How Zhihu Built a Unified Agent Memory Service with Mem0 and MongoDB

Why Chat History Isn't Enough for Agent Memory

As LLMs evolve from single-turn QA to autonomous agents, the core challenge shifts from "can the model answer?" to "can the agent remember past experiences and apply them correctly in future tasks?" At Zhihu, multiple agent types run simultaneously: DBAgent for database diagnostics, Platform Agent for Kubernetes and infrastructure issues, and Algorithm Agent for model and job problems. These agents serve different teams, use different knowledge extraction rules, yet need to share memories within defined scopes. For example, a slow business response might involve a slow SQL found by DBAgent, container CPU jitter detected by Platform Agent, and an inference spike noticed by Algorithm Agent. Only by continuously accumulating these experiences can agents move from isolated problem-solving to joint analysis.

Simply storing full chat histories fails for four reasons: context window bloat with irrelevant tokens; chat logs contain noise, not stable facts like root causes or verified solutions; different agents require different visibility (private vs. shared); and ops scenarios need exact keyword retrieval (error codes like ORA-01652, index names, version numbers) which pure vector search cannot reliably provide. Therefore, a dedicated long-term memory service with extraction, update, isolation, retrieval, and audit capabilities is required.

Mem0 and MongoDB: Complementary Roles

Mem0: Deciding What to Remember

Mem0 acts as the memory layer for AI applications, combining LLM, embedding model, vector database, and memory lifecycle management. A typical memory write is not a raw insert but a pipeline: extract long-term facts from conversation or task results; find potentially related existing memories; decide to ADD, UPDATE, DELETE, or NONE; generate embeddings for the final memories; persist memories and their change history. Mem0's Memory.from_config() allows plugging in different LLMs, embeddings, and vector stores; MongoDB is one supported vector backend. Note that Mem0's write semantics evolve across versions; the project pins Mem0 1.x and enhances its "extract then decide ADD/UPDATE/DELETE/NONE" behavior. Production must lock the Mem0 version and use regression tests to fix memory update semantics.

MongoDB: Storage and Hybrid Retrieval

MongoDB serves multiple roles simultaneously: memory content and metadata storage; embedding vector storage; MongoDB Vector Search for semantic retrieval; MongoDB Search for full-text retrieval; tenant/user/agent namespace filtering; API key, memory table, and tenant model configuration storage. Vector search excels at "semantically similar but differently phrased" queries, while full-text search excels at exact identifiers like error codes, table names, index names. MongoDB officially recommends combining Vector Search and full-text search when both semantic similarity and keyword precision are needed.

Overall Architecture

Agent Consumption Layer

DBAgent, Platform Agent, and Algorithm Agent do not depend on Mem0 SDK or access MongoDB directly. They interact via a unified HTTP API, providing decoupling of agent orchestration from memory implementation, insulation from Mem0 upgrades, centralized credential management for MongoDB/LLM/embedding, and unified enforcement of permissions, rate limits, audit, and data retention policies. DeepAgent handles task orchestration; Agent Memory handles long-term memory; responsibilities are independent.

API Access and Identity Layer

Each caller authenticates via X-API-Key. The server derives tenant_id from the API Key, not trusting tenant claims in the request body. API Keys carry scoped permissions: read (read/search memories), write (write private or agent memories), write:shared (write team-shared memories). Keys can bind fixed user_id or agent_id. In production, real identities should be injected by a trusted gateway, JWT, or mTLS to prevent client-side header forgery.

Memory Space Routing

A request routes to a tenant_id + table + scope tuple. A table represents an independent memory domain (e.g., dba-ops, k8s-incidents, algorithm-jobs). Each non-default table maps to its own MongoDB collection, vector index, full-text index, and fact-extraction prompt. This keeps DBA fault experiences separate from algorithm task experiences. Current implementation queries one table at a time; cross-domain joint retrieval is delegated to the upper agent orchestration layer.

Three-Tier Memory Isolation

Within each table, memories are partitioned into three scopes: private: tenant:<tenant>:user:<user> — user-specific memories (e.g., preferred databases, business lines, personal preferences). shared: tenant:<tenant>:shared — tenant-wide shared memories (confirmed root causes, standard procedures, team ops experience). Readable by all agents; writable only with write:shared permission. agent: tenant:<tenant>:agent:<agent> — agent-internal memories (diagnostic preferences, tool usage patterns, container analysis strategies).

Default retrieval merges private and shared; agent scope is included only when explicitly allowed.

MongoDB Data Organization

Key collections: api_keys: API key hashes, tenant, permissions, identity bindings. tenant_prompts: table registration, physical collection names, extraction prompts. tenant_llm_configs: tenant-level LLM configurations. memories: default memory collection for legacy data. mem_<tenant>_<table>_<hash>: dedicated collections per non-default table.

SQLite history: Mem0 local change history.

MySQL history: optional cross-replica audit dual-write.

Table deletion uses soft delete; collections and data are retained to prevent accidental history loss.

Memory Write Pipeline

Step 1: Auth and Routing

Service parses tenant from API Key, resolves table to physical collection, extraction prompt, and tenant LLM config. Permission checks run before invoking Mem0, preventing unauthorized requests from reaching model or database layers.

Step 2: Fact Extraction

Mem0 uses LLM to extract long-term facts from raw dialogue, not storing full chat logs. Different tables configure different extraction prompts. For DBA table, prompts target: fault time and impact scope; database, table, index, SQL; root cause, evidence, excluded items; executed operations and results; final resolution and follow-up recommendations. Algorithm task tables use separate extraction rules.

Step 3: Long-Text Chunking

Native single-pass fact extraction on long documents can produce incomplete JSON ("API success but zero facts saved"). When input exceeds a threshold, the service: splits by paragraph, newline, and sentence boundaries; parallelizes LLM extraction per chunk; merges facts preserving original order; deduplicates and removes empty facts; feeds the unified fact list to Mem0 for a single memory update decision. If a chunk fails, other chunks' results are still saved, and a degraded response with alert is returned.

Step 4: Memory Update

For each new fact, Mem0 (v1.x behavior) compares against related existing memories and decides: ADD (no similar memory), UPDATE (old memory outdated), DELETE (old memory explicitly contradicted), NONE (duplicate or unnecessary). This avoids repeated accumulation of identical facts, suiting long-running agents.

Step 5: Persistence and Audit

Final memories receive embeddings and are written to MongoDB with content, tenant, scope, source, agent, trace ID, and other metadata. Mem0's local SQLite continues as change history source. If MySQL is configured, the service asynchronously logs: original input messages; tenant, table, namespace; trace ID; old and new memory values; ADD/UPDATE/DELETE events. Audit failures do not block primary memory writes but are logged in runtime metrics.

Memory Retrieval Pipeline

Why Hybrid Retrieval

Vector search suits natural language queries like "Why did the database suddenly slow down?" — it can recall memories phrased as "write pressure increase caused lock waits." Full-text search suits identifier-heavy queries: "Which incident involved ORA-01652?", "Why was idx_member dropped?", "Known issues in v2.3.1?" — exact token matching is more reliable for strong identifiers.

Adaptive Query Planning

The service classifies queries into: prose (natural language dominant), identifier (error codes, table/index names, versions dominant), mixed (both). Each type uses distinct vector and full-text weights instead of a fixed ratio. MongoDB full-text index uses a Chinese analyzer plus custom identifier fields to keep tokens like ORA-01652, member_contacts, idx_member, v2.3.1 intact for matching.

Score Fusion

Vector similarity scores and BM25 scores are on different scales. The service normalizes each separately, then applies weighted fusion. Default uses application-layer score fusion; RRF and MongoDB $rankFusion modes are retained. Returned results include score, vector_score, text_score, matched fields, letting callers distinguish semantic hits, keyword hits, or dual hits.

Retrieval Degradation

If full-text index is not ready, hybrid falls back to vector search; if vector index is not ready, falls back to keyword search. On custom hybrid search exceptions, the service attempts fallback to Mem0's built-in vector search, preventing a single retrieval path failure from making memory completely unavailable.

Production Reliability Design

Read/Write Resource Isolation

Writes require multiple LLM calls and have higher latency than reads. Separate concurrency budgets for read and write requests prevent slow writes from starving retrieval threads.

Write Admission Control

Based on recent write latency P95, current executing count, queue length, and request budget, the service estimates if a new request can complete before gateway timeout. If not, it immediately returns HTTP 429 with Retry-After, which is easier for callers to retry correctly than queuing for tens of seconds and returning 504.

Timeouts, Retries, Error Classification

LLM and embedding calls get explicit timeouts, exponential backoff retries, classification of 429/502/503/504, query embedding TTL caching, and single-flight merging for identical queries. The service also verifies Mem0's actual write result: if upstream failed and zero memories were saved, it does not falsely report success to the caller.

Memory Space Caching

Each tenant + table builds a distinct Memory Space containing: Mem0 instance, MongoDB collection, vector and full-text index managers, hybrid retriever, tenant LLM config fingerprint. These objects are reused via LRU cache. First-time construction for a Memory Space uses a dedicated lock to avoid cold-start stampedes under concurrent requests.

Observability

The service records latency for API, LLM, embedding, MongoDB, fact extraction, memory update stages, exportable via StatsD/Graphite per tenant and operation type. This enables answering: which tenant writes most; whether LLM or MongoDB drives latency; which stage write failures occur in; which table indexes are not READY; which retrieval method contributed to final results.

Architectural Boundaries

The system is a long-term memory service, not a full knowledge base or RAG platform replacement. Explicit boundaries:

Single-table query per retrieval; cross-domain retrieval handled by agent orchestration.

Write remains synchronous; high-scale adoption should move to async tasks.

Tenant LLM secrets need integration with Secret Manager or encrypted storage.

Mem0 dependency requires lockfiles to prevent inconsistent 1.x versions across deployments.

Memory quality cannot be measured by "write success" alone; continuous evaluation of recall rate, error memory rate, conflict rate, and expired memory ratio is needed.

Future work can add a Memory Refinery: periodic scan of low-value, long-unhit, conflicting, or expired memories for merging, down-weighting, archiving, and human review.

Summary

In this Agent Memory architecture, three components own distinct responsibilities:

Mem0: understands dialogue, extracts facts, manages memory updates.

MongoDB: stores memories, provides vector search, full-text search, metadata filtering.

Agent Memory Service: multi-tenant isolation, multi-table routing, three memory scopes, permissions, reliability, audit, observability.

It is not merely an HTTP wrapper around Mem0; it extends a single-app memory SDK into unified infrastructure serving DBAgent, Platform Agent, Algorithm Agent, and other business agents simultaneously. For multi-agent systems, the real value is not "how many conversations saved" but whether each task's effective experience can be precipitated, retrieved at the right time, in the right scope, by the right agent. When agents gain accumulable, searchable, updatable, auditable long-term memory, multi-agent collaboration acquires a foundation for continuous evolution.