AI Agent Memory Stack

Designing Memory for Enterprise Agents

📌1. Working Memory

Definition:

Temporary memory that holds information within a limited context window.
Operates on a FIFO (First-In-First-Out) sliding window principle.

Process:

Query → Context Window (Turn n-4, Turn n-3, Turn n-2, Turn n-1, Turn n) → LLM → Answer
Older turns are dropped as new ones are added to the window.

📌2. Episodic Memory

Definition:

Stores information about specific events or experiences, retrievable on demand.

Process:

Query → Embedder → Vector Store → Top-K Episodes → Context Assembly → LLM → Answer
New episodes are stored for future retrieval.

📌3. Semantic Memory

Definition:

Contains distilled facts and structured knowledge, representing what the agent knows to be true.

Process:

Query → Entity Extractor → Knowledge Graph/Structured Facts → Fact Lookup → Context Assembly → LLM → Answer

📌4. Procedural Memory

Definition:

Encodes skills and workflows, enabling the agent to perform tasks.

Process:

Query → Skill Matcher → Skill Library (e.g., invoice_approval, data_cleanup, report_gen) → Execute Workflow → Answer

📌5. Hierarchical Memory

Definition:

A tiered memory system inspired by operating systems, with hot, warm, and cold storage layers.

Process:

Query → Working Memory (hot) → Episodic Memory (warm) → Archival Memory (cold) → LLM → Answer

📌6. Prospective Memory

Definition:

Handles reminders, retries, follow-ups, and autonomy for remembering what comes next.

Process:

Task/Intent → Scheduler/Trigger Engine → Task Queue → Deferred Action → Execution → Result

📌7. Shared Memory

Definition:

Ensures consistency and alignment across multiple agents by maintaining a shared truth.

Process:

Agents (A, B, C, …, N) → Shared Memory Layer (State Store/Knowledge Bus) → Verified State, Consistency, Conflict Resolution, Access Control

📌8. Flowchart Representation

QUERY
↷
CONTEXT WINDOW
↷
LLM
↷
ANSWER QUERY
↷
EMBEDDER
↷
VECTOR STORE
↷
TOP-K EPISODES
↷
CONTEXT ASSEMBLY
↷
LLM
↷
ANSWER QUERY
↷
ENTITY EXTRACTOR
↷
KNOWLEDGE GRAPH
↷
FACT LOOKUP
↷
CONTEXT ASSEMBLY
↷
LLM
↷
ANSWER QUERY
↷
SKILL MATCHER
↷
SKILL LIBRARY
↷
EXECUTE WORKFLOW
↷
ANSWER QUERY
↷
WORKING MEMORY
↷
EPISODIC MEMORY
↷
ARCHIVAL MEMORY
↷
LLM
↷
ANSWER TASK/INTENT
↷
SCHEDULER
↷
TASK QUEUE
↷
DEFERRED ACTION
↷
EXECUTION
↷
RESULT AGENTS
↷
SHARED MEMORY LAYER
↷
VERIFIED STATE
↷
CONSISTENCY
↷
CONFLICT RESOLUTION
↷
ACCESS CONTROL

📌9. Real-Time Story: 🛠 Building an AI Assistant

Imagine you're designing an AI assistant for a customer service team. Here's how the memory stack works in practice:

Working Memory: The assistant processes the current conversation with a customer, keeping track of the last few exchanges to provide relevant responses.
Episodic Memory: It retrieves past interactions with the same customer, such as complaints or feedback, to personalize its answers.
Semantic Memory: The assistant accesses company policies and FAQs to ensure its responses are accurate and consistent.
Procedural Memory: When asked to process a refund, the assistant uses its encoded workflows to execute the task efficiently.
Hierarchical Memory: The assistant prioritizes recent conversations (hot memory) while archiving older ones (cold memory) for long-term storage.
Prospective Memory: It schedules follow-ups and reminders for unresolved issues, ensuring no task is forgotten.
Shared Memory: Multiple assistants working across departments share a unified knowledge base, ensuring consistent communication and decisions.

This layered memory system enables the AI assistant to operate effectively, adapt to dynamic situations, and provide a seamless user experience.