2026-06-10

What is agent memory (short-term vs long-term)?

Agent memory is the information an AI agent keeps so it can keep working across turns, and it usually means two different things: short-term memory for the current task context, and long-term memory for stored facts or past interactions it can retrieve later.

Why it matters

Without memory, an agent is basically reset every time you ask it something new. That makes it bad at multi-step tasks, personalization, follow-up questions, and anything that needs continuity.

In practice, memory helps an agent:

remember what the user just said
track task state across steps
avoid repeating work
reuse useful facts from past sessions
act more consistently over time

Most teams start with short-term memory because it is simpler and safer, then add long-term memory only when they need persistence across sessions.

How it works

Short-term memory

Short-term memory is the agent’s working context for the current interaction. In LLM systems, this is usually the conversation history, tool outputs, system instructions, and any task state that is still relevant.

It is limited by the model’s context window and by practical prompt design. When the context gets too large, older or less relevant information may be truncated, summarized, or retrieved selectively.

Long-term memory

Long-term memory is information the agent stores outside the model so it can use it later. This is often implemented with a database, document store, vector store, or key-value memory system.

Typical long-term memory content includes:

user preferences
saved facts
task histories
summaries of prior conversations
reusable domain knowledge

The agent does not “remember” this on its own; it must retrieve the stored item and feed it back into the model at the right time.

The key distinction

Short-term memory is for working on the current task. Long-term memory is for persisting useful information beyond the current context.

A common architecture is:

keep the current conversation in short-term context
write selected facts or summaries into long-term storage
retrieve only the relevant memories when needed

That selective retrieval matters, because dumping everything into the prompt usually hurts quality more than it helps.

Tiny concrete example

User: “Plan a 3-day trip to Tokyo. I like sushi and walking tours.”

Short-term memory: the agent keeps “3-day Tokyo trip,” “sushi,” and “walking tours” in the current conversation.
Long-term memory: after the chat, it stores “user likes sushi and walking tours.”

A week later:

User: “Plan my next trip.”

The agent can retrieve the stored preference and say:

Assistant: “You previously said you like sushi and walking tours, so I’ll bias the itinerary toward those.”

Common pitfalls / when NOT to use it

Do not store everything. Long-term memory should be selective; saving raw chat logs as “memory” is usually noisy and risky.
Do not trust memory as truth. User preferences can change, stored facts can be outdated, and retrieved items can be wrong or irrelevant.
Do not use long-term memory for secrets unless you have a clear security and consent model.
Do not over-rely on short-term context. If a task must survive a restart or a later session, you need persistent storage, not just conversation history.
Do not confuse memory with reasoning. Memory helps the agent recall information; it does not guarantee better judgment.

For most applications, start with a clean short-term context strategy, then add long-term memory only for clearly valuable, user-visible continuity.