Agent Memory in 2026: Short-Term vs Long-Term vs Episodic

TL;DR

• Memory transforms stateless models into stateful products that learn and adapt
• Three core memory types: short-term (current task), long-term (knowledge base), episodic (past decisions and outcomes)
• Modern architectures use temporal knowledge graphs that outperform traditional RAG
• ENGRAM achieves state-of-the-art results using only ~1% of tokens vs. full-context baselines
• The biggest memory mistake is saving everything — you want the smallest memory that improves outcomes
• Store decisions as records, not raw chat logs

---

Why Memory Matters

Without memory, every interaction starts from zero. The agent:

• Forgets past conversations
• Can’t learn from mistakes
• Repeats the same errors
• Can’t build on previous work
• Has no context about the user

Memory is what turns a model into a product.

The Memory Hierarchy

Memory Type	What It Stores	Duration	Example
Short-term	Current task context	Single session	”User wants to refund order #12345”
Long-term	Knowledge and facts	Persistent	Product documentation, policies
Episodic	Past decisions and outcomes	Persistent	”Last time we tried X, it failed”
Procedural	Learned skills	Persistent	Successful workflow patterns
Semantic	General knowledge	Persistent	Domain concepts and relationships

---

Short-Term Memory (Context)

Short-term memory holds the immediate context for the current task.

What to Store

Element	Purpose
Current user goal	What are we trying to accomplish?
Immediate constraints	What limits apply right now?
Tool outputs in current run	Results from recent tool calls
Conversation history	Recent turns of dialogue
User state	Current permissions, preferences

Implementation Approaches

Approach	Trade-off
Full context window	Simple but expensive, limited by window size
Sliding window	Loses early context
Summarization	Loses detail
Selective extraction	Requires good extraction logic

Best Practices

Practice	Rationale
Keep it small	Only what’s needed for current task
Keep it structured	JSON/schema over raw text
Clear irrelevant context	Remove when task changes
Prioritize recent turns	Most recent = most relevant

Example: Structured Short-Term Memory

{
  "session_id": "sess_abc123",
  "user_goal": "Get refund for damaged item",
  "constraints": {
    "order_id": "12345",
    "order_date": "2026-01-15",
    "refund_window": "30 days"
  },
  "tool_outputs": [
    {
      "tool": "get_order_status",
      "result": {"status": "delivered", "damage_reported": true}
    }
  ],
  "conversation_summary": "User received damaged item, wants refund"
}

---

Long-Term Memory (Knowledge)

Long-term memory stores persistent knowledge that applies across sessions.

What to Store

Content	Purpose
Documentation	Product info, how-tos
Policies	Business rules, constraints
FAQs	Common questions and answers
Domain knowledge	Industry concepts, terminology
Entity information	Products, customers, transactions

Implementation: Beyond Basic RAG

Traditional RAG retrieves from static documents. Modern approaches must handle:

Challenge	Solution
Evolving data	Continuous indexing
User interactions	Dynamic memory updates
Multimodal sources	Unified representation
Freshness requirements	Temporal awareness

Temporal Knowledge Graphs

The Zep architecture uses a temporally-aware knowledge graph engine (Graphiti) that:

• Synthesizes conversational and business data
• Maintains historical relationships
• Outperforms MemGPT on benchmarks (94.8% vs 93.4%)

Index Quality > Embedding Hype

The critical factors for long-term memory:

Factor	Impact
Index quality	What you put in determines what you get out
Freshness	Stale knowledge = wrong answers
Chunking strategy	Affects retrieval precision
Metadata	Enables filtering and prioritization

Best Practices

Practice	Rationale
Structured indexing	Better retrieval than raw text
Regular updates	Knowledge drifts over time
Source attribution	Know where information came from
Version control	Track changes over time

---

Episodic Memory (Experience)

Episodic memory stores specific past events, decisions, and outcomes.

What to Store

Element	Purpose
Past decisions	What did we decide?
Outcomes	What happened?
User preferences	What does this user like?
Workflow history	What steps were taken?
Failure patterns	What went wrong before?

Why Episodic Memory Matters

Benefit	Example
Learn from mistakes	”We tried X and it failed”
Personalization	”User prefers Y”
Efficiency	Skip known-bad approaches
Continuity	Resume where we left off

The AriGraph Pattern

Modern episodic memory integrates with semantic memory through graph structures:

• Episodic nodes capture specific events
• Semantic nodes capture general knowledge
• Relationships connect events to concepts
• Enables complex reasoning across both

Storing Decisions, Not Logs

The key insight: store decisions as records, not raw chat logs.

Don’t Store	Do Store
”User said: can you help me…”	Decision: initiated refund workflow
Full conversation transcript	Outcome: refund approved for $47.99
Every intermediate step	Key decision points with rationale
Tool call raw responses	Tool results relevant to decision

Example: Episodic Record

{
  "episode_id": "ep_789",
  "timestamp": "2026-01-27T14:30:00Z",
  "user_id": "user_456",
  "context": "Order refund request",
  "decision": "Approved refund without manager approval",
  "rationale": "Within policy limits, verified damage",
  "outcome": "Success",
  "tools_used": ["get_order_status", "initiate_refund"],
  "learnings": [
    "Damage verification photo expedited approval"
  ]
}

---

Multi-Type Memory Architectures

Modern systems organize multiple memory types through coordinated systems.

ENGRAM Architecture

ENGRAM organizes conversations into three canonical memory types through a single router and retriever:

• Achieves state-of-the-art results on LongMemEval
• Uses only ~1% of tokens compared to full-context baselines
• Dense retrieval replaces complex multi-stage pipelines

MIRIX Architecture

MIRIX extends the memory framework with six distinct types:

• Short-term (working memory)
• Long-term (persistent knowledge)
• Episodic (specific events)
• Semantic (general knowledge)
• Procedural (learned skills)
• Meta-memory (memory about memory)

A multi-agent framework coordinates these types for optimal retrieval.

MemVerse: Hierarchical Retrieval

MemVerse combines:

• Fast parametric recall (in-model)
• Hierarchical retrieval-based memory (external)
• Knowledge graphs for organization
• Periodic distillation to compress essential knowledge

Choosing an Architecture

If You Need	Consider
Simple, single-purpose agent	Basic RAG + short-term context
Multi-turn conversation	Episodic + short-term
Knowledge-heavy domain	Long-term + semantic
Personalization	Episodic per-user
Complex workflows	All memory types coordinated

---

The Biggest Memory Mistake

Saving everything.

More memory doesn’t mean better performance. It often means:

• Slower retrieval
• Irrelevant context
• Higher costs
• Confusion between old and new information

The Principle

You want the smallest memory that improves outcomes.

Memory Hygiene

Practice	Implementation
Expiration	Old memories fade or delete
Consolidation	Compress many episodes into patterns
Pruning	Remove low-value entries
Prioritization	Rank by relevance to current task

Deciding What to Remember

Remember	Forget
Decisions with outcomes	Routine confirmations
User preferences	Transient state
Errors and failures	Successful standard workflows
Policy-relevant facts	Timestamp details
Relationships	Intermediate calculations

---

Implementation Patterns

Pattern 1: Session-Scoped Memory

For stateless or low-context applications:

Start session
  ↓
Load user profile (if exists)
  ↓
Maintain context through session
  ↓
Save key decisions/preferences
  ↓
Clear working memory
  ↓
End session

Pattern 2: Continuous Memory

For agents that learn over time:

Every interaction:
  ↓
Retrieve relevant long-term + episodic
  ↓
Add to short-term context
  ↓
Process interaction
  ↓
Extract learnings
  ↓
Update episodic memory
  ↓
(Periodically) Consolidate to long-term

Pattern 3: Workspace Memory

For agents working on projects/artifacts:

Start work session
  ↓
Load project state from memory
  ↓
Track changes during session
  ↓
Checkpoint periodically
  ↓
Save final state with summary
  ↓
Index for future retrieval

---

Memory and RAG Integration

Evolution of RAG

Generation	Approach	Limitation
Gen 1	Static document retrieval	No personalization, no learning
Gen 2	User-specific retrieval	Limited by document boundaries
Gen 3	Dynamic memory + retrieval	Complexity, coordination overhead
Gen 4	Graph-based unified memory	Current state-of-the-art

When to Use What

Use Case	Approach
Static documentation	Traditional RAG
User preferences	Episodic memory
Dynamic business data	API tools + caching
Cross-session learning	Knowledge graph
Real-time context	Short-term memory

---

Implementation Checklist

Before building:

• Define what needs to be remembered
• Classify by memory type (short/long/episodic)
• Define retention policies
• Plan data schema for each type

Short-term memory:

• Define context structure
• Implement context window management
• Set up context clearing logic
• Handle session boundaries

Long-term memory:

• Set up vector store or knowledge graph
• Define chunking strategy
• Implement retrieval pipeline
• Plan update/refresh process

Episodic memory:

• Define episode schema
• Implement decision extraction
• Set up outcome tracking
• Create consolidation process

Operations:

• Set up monitoring for memory size/quality
• Define pruning/expiration policies
• Plan backup and recovery
• Test retrieval accuracy regularly

---

FAQ

What’s the biggest memory mistake?

Saving everything. You want the smallest memory that improves outcomes. More data ≠ better performance — it often means slower retrieval, more confusion, and higher costs.

How long should memories persist?

Memory Type	Typical Duration
Short-term	Session only
User preferences	Indefinite
Episodic events	Months, with consolidation
Knowledge base	Until superseded

Should I use a vector database or knowledge graph?

Use Vector DB When	Use Knowledge Graph When
Simple retrieval	Complex relationships matter
Document-based knowledge	Entity-centric knowledge
Similarity search is primary	Traversal queries needed
Lower complexity acceptable	Relationship reasoning required

How do I handle conflicting memories?

• Use timestamps to prefer recent
• Track confidence/certainty
• Implement explicit override mechanisms
• Consolidate periodically to resolve conflicts

How much memory is too much?

Monitor these signals:

• Retrieval latency increasing
• Irrelevant results in top-k
• Storage costs growing faster than value
• Agent confusion from contradictory context

When these appear, prune and consolidate.

How do I test memory systems?

Test Type	What It Validates
Retrieval precision	Right memories found
Retrieval recall	All relevant memories found
Conflict resolution	Contradictions handled
Expiration	Old memories fade correctly
Performance	Speed under load

---

Sources & Further Reading

• Zep: A Temporal Knowledge Graph Architecture for Agent Memory — arXiv
• MemVerse: Multimodal Memory for Lifelong Learning Agents — arXiv
• ENGRAM: Multi-type Memory System — arXiv
• MIRIX: Multi-Agent Memory System for LLM-Based Agents — arXiv
• Why Chatbots Are Dead: The Era of Agentic Workflows
• RAG vs Fine-Tuning vs Tool Use in 2026
• Agent Evaluation Harnesses in 2026