AskTable Permanent Memory System: Let AI Remember Every Conversation

Every conversation is an island - this is the pain point of traditional AI assistants. User-stated preferences, corrected mistakes, and established context are all lost in the next conversation. AskTable's newly introduced permanent memory system is breaking this困局.

---

I. The Nature of the Problem

1.1 The Memory Dilemma of Conversational AI

┌─────────────────────────────────────────┐
│         Conversation 1                   │
│  User: What's East China sales?          │
│  AI: 12 million                         │
│  User: For what time period? From now on │
│        when answering sales metrics,     │
│        include specific time             │
│  AI: Got it. Q1 2026 East China sales  │
│      is 12 million                      │
└─────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────┐
│         Conversation 2 (any time later)   │
│  User: How's East China profit?          │
│  AI: Q1 2026 net profit 3.5 million,   │
│      YoY +18%, QoQ +5%               │  ← Automatically with time
└─────────────────────────────────────────┘

Root cause: In traditional architecture, conversation history is only valid within a single session.

1.2 Fragmented User Experience

This "amnesia" leads to:

• •Repeated explanations: Must re-describe preferences every time
• •Context discontinuity: AI can't understand user's historical behavior patterns
• •Low intelligence: Seems smart but starts from scratch every time

---

II. Technical Solution Selection

2.1 Why Choose mem0?

AskTable chose mem0 OSS as the memory framework, core reasons:

2.2 Vector Storage: Reusing Qdrant

┌─────────────────────────────────────────────────────────────┐
│                    AskTable Vector Storage Architecture       │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│   ┌──────────────┐        ┌──────────────┐                 │
│   │   Qdrant      │        │   Qdrant      │                 │
│   │  (existing)   │        │  (memory-    │                 │
│   │  metadata     │        │   dedicated) │                 │
│   │  retrieval    │        │  cross-session│                 │
│   │               │        │  memory       │                 │
│   └──────────────┘        └──────────────┘                 │
│                                                             │
│   Shared infrastructure, reduced operations cost             │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Reuses existing Qdrant instance, no new infrastructure dependencies introduced.

---

III. System Architecture

3.1 Overall Flow

┌─────────────────────────────────────────────────────────────┐
│                    Memory System Workflow                     │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  Conversation starts                                          │
│      │                                                        │
│      ▼                                                        │
│  ┌─────────────────┐                                        │
│  │  Memory Search  │  ← Search related memories with query    │
│  └────────┬────────┘                                        │
│           │                                                  │
│           ▼                                                  │
│  ┌─────────────────┐                                        │
│  │  Format and inject│  ← Splice into System Prompt          │
│  │  memory_context  │                                        │
│  └────────┬────────┘                                        │
│           │                                                  │
│           ▼                                                  │
│  ┌─────────────────┐                                        │
│  │  Agent generates │  ← Carry memory context                │
│  │  response        │                                        │
│  └────────┬────────┘                                        │
│           │                                                  │
│           ▼                                                  │
│  ┌─────────────────┐                                        │
│  │  Memory Write   │  ← Async write after conversation ends  │
│  └─────────────────┘  (fire-and-forget)                      │
│                                                             │
└─────────────────────────────────────────────────────────────┘

3.2 Read-Write Separation Design

Read (Search) at request start:

• •Semantic search with user query
• •Take Top-K related memories
• •Inject into System Prompt

Write (Add) after response completes:

• •Async execution, non-blocking response
• •mem0 internally calls LLM to extract structured facts
• •Automatic deduplication and conflict resolution

3.3 Isolation Granularity: Data Agent Level

┌─────────────────────────────────────────────────────────────┐
│                    Memory Isolation Model                     │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│   DataAgent-A                                               │
│   ┌─────────────────────────────────┐                      │
│   │ Memory space: agent_id = "agent-a"│                      │
│   │ - User preferences               │                      │
│   │ - Common reports                 │                      │
│   │ - Analysis habits                │                      │
│   └─────────────────────────────────┘                      │
│                                                             │
│   DataAgent-B (independent memory)                           │
│   ┌─────────────────────────────────┐                      │
│   │ Memory space: agent_id = "agent-b"│                      │
│   │ - Different user preferences      │                      │
│   │ - Different business scenarios    │                      │
│   └─────────────────────────────────┘                      │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Design principle: All users under the same Data Agent share memory (team knowledge base mode).

---

IV. Core Implementation

4.1 MemoryService Abstraction Layer

class MemoryService(Protocol):
    """Memory service abstraction, supports later replacement"""

    async def add(
        self,
        messages: list[dict],
        agent_id: str,
        metadata: dict | None = None
    ) -> list[str]: ...

    async def search(
        self,
        query: str,
        agent_id: str,
        top_k: int = 5
    ) -> list[MemoryItem]: ...

    async def get_all(
        self,
        agent_id: str
    ) -> list[MemoryItem]: ...

    async def delete(
        self,
        memory_id: str
    ) -> None: ...

Value: Through Protocol abstraction, ensuring mem0 is just one implementation, can smoothly switch to custom solution later.

4.2 Memory Context Formatting

# Extract key info from memory, format as natural language
MEMORY_CONTEXT_TEMPLATE = """
## Historical Memory
{relevance_memories}

Here are historical information relevant to you. Please answer user questions combined with this context.
"""

Example output:

## Historical Memory
- When user asks about sales, profit margin and other metrics, should include specific time period (2026-04-01)
- User mainly focuses on East China region data (2026-03-15)
- Prefers showing YoY and QoQ growth rates in charts (2026-03-20)

Here are historical information relevant to you. Please answer user questions combined with this context.

4.3 Async Write Implementation

async def run_conversation_task(ctx: Context, ...):
    # ... Execute conversation ...

    # After releasing lock, async write memory
    await release_lock()
    await stream_end()

    # Fire-and-forget, doesn't block user receiving response
    asyncio.create_task(
        memory_service.add(
            messages=full_conversation,
            agent_id=data_agent_id
        )
    )

---

V. Product Value

5.1 User Experience Upgrade

Before (no memory):

User: What's East China sales?
AI: 12 million
User: For what time period?
AI: This is Q1 data

User (another day): How's East China profit?
AI: Q1 2026 net profit 3.5 million
    [No time dimension, don't know which time period]

After (has memory):

User: What's East China sales?
AI: 12 million
User: For what time period? From now on when answering sales metrics, should include specific time
AI: Got it. Q1 2026 East China sales is 12 million

User (another day): How's East China profit?
AI: Q1 2026 net profit 3.5 million, YoY +18%, QoQ +5%
    ← Automatically with time dimension, no need to repeat

5.2 Intelligence Scenarios

5.3 Frontend Interaction

The system added a "Memories" Tab to the Data Agent configuration page:

• •Switch control: Enable/disable memory function
• •Memory list: View all memories of current Agent
• •Manual add: Supplement important preferences
• •Search memory: Search specific memories
• •Delete memory: Remove unwanted memories

---

VI. Technical Challenges and Responses

6.1 LLM Call Costs

Challenge: mem0 needs 1-2 LLM calls per add() for fact extraction.

Response:

• •Write async, non-blocking response
• •Batch processing, accumulate multi-turn conversations then write together
• •Enable on-demand (through memory_enabled switch)

6.2 Chinese Scenario Quality

Challenge: mem0's fact extraction prompt mainly targets English.

Response:

• •Evaluation shows AskTable memories are relatively structured (table names, metric names)
• •This type of memory extraction is relatively simple, Chinese-English differences have small impact
• •Continuously monitor extraction quality, adjust prompt if necessary

6.3 Debugging Difficulty

Challenge: When memory errors occur, need to trace mem0 internal链路.

Response:

• •Complete logging
• •Frontend visible memory list, convenient for manual verification
• •Protocol abstraction ensures replaceability

---

VII. Comparison with Industry Solutions

AskTable's choice: mem0 is the "fast validation" path, Protocol abstraction leaves room for "long-term evolution".

---

VIII. Future Evolution

8.1 Potential Optimization Directions

• •Memory classification: Important vs temporary memories, set different expiration times
• •Proactive memory: Agent proactively judges whether to write memory
• •Memory visualization: Graphically display memory relationships
• •Memory evaluation: Quantify memory's effect improvement

8.2 Custom Development Path

If mem0 performs poorly in Chinese data analysis scenarios, can switch to custom solution:

Existing infrastructure:
├── Qdrant (vector storage)
├── BAAI/bge-small-zh-v1.5 (Embedding)
└── Langfuse (observability)

Custom development needs to solve:
├── Fact extraction Prompt
├── Deduplication/conflict strategy
└── Memory decay mechanism

---

IX. Summary

AskTable permanent memory system's introduction marks AI data analysis assistant's evolution from "reactive tool" to "intelligent partner":

Core value:

1. •Context continuity: Remember user preferences across sessions
2. •Active learning: Automatically extract valuable information from conversations
3. •Personalized service: Each interaction understands user better than the last

Technical highlights:

1. •mem0 + Qdrant combination, balancing development efficiency and long-term maintainability
2. •Protocol abstraction ensures no technology lock-in
3. •Read-write separation design, balancing performance and effectiveness

Product significance: Memory system lets AI truly "recognize" users, not just answer questions. This is the key step from "tool" to "assistant".

---

Related Reading

• •AskTable AI Agent Working Principles
• •AskTable Skill System