Enterprise MultiAgent Memory: Short-Term Context & Four-Layer Architecture

Background and Overall Architecture

In a MultiAgent platform, an agent request may traverse models, MCP/A2A tools, RAG, workflows, and sandboxes, while needing to remember user preferences, task progress, and collaboration agreements across multi-turn dialogues and cross-session cooperation. Therefore, the memory module is not an independent add-on but part of the agent execution chain.

The system addresses three needs: session continuation (short-term history for reference resolution and task continuation), cross-session reuse (user preferences and stable facts), and context association (linking new messages to existing context).

MemOS was selected after evaluation on 1,540 questions with 10 test users, achieving a composite score of 74.33% (single-hop and multi-hop performance met thresholds; evaluation for selection reference only, not production SLA).

Overall Architecture

The backend uses Spring Boot 3 and Java 17. Agent orchestration adopts AgentScope. New agents default longMemoryProvider to MEMOS, but openLongMemory is off by default. When enabled, short-term session history and long-term memory load in parallel at request start; after session end, onSessionEndAsync asynchronously handles filtering, deduplication, and long-term persistence. Short-term memory uses MySQL for persistence and Redis as hot cache; long-term memory uses MemOS as primary path, with MySQL/Mem0 routing as fallback.

Four-Layer Memory Model

The four layers correspond to different lifecycles:

Working Memory – serves only the current reasoning step.

Session Memory – stores message history within a session.

User Memory – records cross-agent shared preferences and stable facts, mapped to MemOS user_profile.

Agent Memory – stores task experience and collaboration conventions for a specific agent, mapped to MemOS agent_{agentId}.

After a session ends, new messages are judged and deduplicated, then precipitated from the Session layer to User or Agent layers.

The four-layer model and MemOS memory types ( text_mem, pref_mem, skill_mem, tool_mem) are two classification dimensions: the former describes lifecycle and scope, the latter describes content shape; they do not map one-to-one.

Routing behavior: if longMemoryProvider is unconfigured, agent missing, or config read fails, MemoryRpcServiceImpl falls back to MySQL; explicit MEMOS or Mem0 configuration enters the corresponding provider. MemOS query exceptions are logged and return empty results without automatic fallback to MySQL.

Memory Loading Flow

A request's memory path: AgentExecutor#execute parses context parameters, then in memoryLoadExecutor launches short-term and long-term memory futures in parallel. Short-term path reads ConversationMemory; long-term path calls queryMemory, with MemoryRpcServiceImpl selecting MEMOS, Mem0, or MySQL per agent config.

Long-term retrieval results are grouped by user_profile and agent scopes, filtered for sensitivity and low relevance, and limited per scope.

A fixed token budget (default 4000 tokens) is allocated: user profile gets up to 60%; remaining budget goes to agent memory.

Final results are written into AgentContext alongside short-term messages for subsequent model calls; after session ends, async long-term precipitation and context summarization are triggered.

Implementation boundary: ConversationMemory handles session history reading and window trimming; AgentScope Harness state store persists session state (shared workspace files with sandbox, in-memory fallback without). The main ModelInvoker chain constructs Harness messages using the current userMsg as entry, so Redis/MySQL history is not unconditionally concatenated into every AgentScope prompt.

Short-Term Memory Loading Details

Short-term memory (Session Memory) stores chronologically ordered dialogue history for the current session, providing direct context for reference resolution and multi-turn reasoning. Key concerns: read latency, Redis/MySQL fallback, and token window control.

Call Chain

AgentExecutor

gets AgentConfig → ModelBindConfigDto → contextRounds. AgentExecutor calls chatMemory.get(conversationId, contextRounds * 3). chatMemory is ConversationMemory interface, implemented by ConversationApplicationServiceImpl.

Implementation reads Redis first, falls back to MySQL on failure.

Key Implementation

// DTO converted to Message, system/tool messages split
trimAlternationFromEnd(chatMsgs);
int windowSize = MAX_TOKEN_WINDOW_SIZE - SUMMARY_TOKEN_BUDGET;
int totalTokens = 0, recentTokens = 0;
boolean overLimit = false;
List<Message> recentWindow = new ArrayList<>();
for (int i = chatMsgs.size() - 1; i >= 0; i--) {
    Message message = chatMsgs.get(i);
    int tokens = TikTokensUtil.tikTokensCount(message.getText());
    totalTokens += tokens;
    if (!overLimit && recentTokens + tokens <= windowSize) {
        recentWindow.add(0, message);
        recentTokens += tokens;
    } else {
        overLimit = true;
    }
}
if (totalTokens <= MAX_TOKEN_WINDOW_SIZE) {
    removeLeadingAssistant(chatMsgs);
    return mergeSystemAndChat(systemMsgs, chatMsgs);
}
removeLeadingAssistant(recentWindow);
String summaryMd = iMemoryRpcService.getConversationSummaryMd(Long.parseLong(cid));
if (StringUtils.isNotBlank(summaryMd)) {
    recentWindow.add(0, new SystemMessage(summaryMd));
}
return recentWindow;

Redis Priority Read

ConversationApplicationServiceImpl#getMessagesFromCache

reads Redis List, converts JSON to ChatMessageDto, reverses to oldest-first; on miss or exception, queries MySQL, then leftPush back to Redis with CACHE_TTL_SECONDS, and uses rightPop to evict oldest messages per MAX_CACHED_MESSAGES. Per-message parsing and exception logging are defensive code.

Redis Storage Structure

Uses leftPush for writes (newest at index 0). Current implementation reads entire list via range(0, Long.MAX_VALUE) and performs window trimming in application layer. The lastN parameter of ConversationMemory#get(String, int) does not participate in Redis range boundary calculation; actual read range is determined by Redis list content and subsequent token window. Redis serves as hot cache only; MySQL remains cold-start fallback and persistence guarantee.

Token Window Control Strategy

The algorithm iterates messages from newest to oldest, accumulating tokens until the window budget (max tokens minus summary budget) is reached. If total tokens fit within the max window, leading assistant messages are removed and system+chat messages merged. Otherwise, the recent window is used, leading assistant messages removed, and a conversation summary (if available) is prepended as a system message.

Short-Term Memory Writing Details

Dual-write significance: MySQL provides persistence guarantee against data loss; Redis provides high-performance reads for real-time dialogue. Fallback: Redis failure still allows MySQL read fallback.

messages.forEach(message -> {
    String conversationId0 = conversationId;
    if (conversationId0.startsWith("agent:")) {
        // Sub-agent non-ChatMessage not written to main session to avoid virtual session pollution.
        if (message instanceof ChatMessageDto) {
            conversationId0 = conversationId.replace("agent:", "");
        } else {
            return;
        }
    }
    ChatMessageDto chatMessage = ...; // type conversion, text cleaning, tenant fields
    ConversationMessage conversationMessage = ...;
    // Write MySQL first; Redis is only a volatile hot cache.
    Long messageId = TenantFunctions.callWithTenantId(chatMessage.getTenantId(),
        () -> conversationDomainService.addConversationMessage(conversationMessage));
    chatMessage.setIndex(messageId);
    try {
        String key = generateConversationKey(conversationId0);
        redisUtil.leftPush(key, JSON.toJSONString(chatMessage));
        redisUtil.expire(key, CACHE_TTL_SECONDS);
        long size = redisUtil.size(key);
        // Newest at head; evict oldest from tail when exceeding 200 messages.
        if (size > MAX_CACHED_MESSAGES) {
            redisUtil.rightPop(key);
        }
    } catch (Exception e) {
        // Cache failure does not roll back completed MySQL write; read will fall back to DB.
        log.warn("Failed to cache message to Redis, conversationId={}", conversationId0, e);
    }
});

After short-term messages are written, onSessionEndAsync checks session summary: triggers summary generation only when messages not covered by summary reach 20; summary injection budget ~2000 tokens, cached in Redis for 1 hour. This separates high-frequency dialogue writes from low-frequency summary merging, avoiding summary model calls every turn.

Long-Term Memory Loading Implementation

Parallel Loading Design

In AgentExecutor#execute, CompletableFuture runs short-term and long-term loading in parallel on a dedicated memoryLoadExecutor thread pool (avoiding ForkJoinPool.commonPool contention). Long-term memory does not block short-term; it queries with empty context first.

final int finalContextRounds = contextRounds;
CompletableFuture<List<Message>> contextMessagesFuture = CompletableFuture.supplyAsync(() -> {
    if (finalContextRounds <= 0) {
        return new ArrayList<Message>();
    }
    return new ArrayList<>(
        chatMemory.get(agentContext.getConversationId(), finalContextRounds * 3));
}, memoryLoadExecutor);

CompletableFuture<Map<String, String>> longMemoryFuture = CompletableFuture.supplyAsync(() -> {
    if (agentContext.getAgentConfig().getOpenLongMemory() != AgentConfig.OpenStatus.Open) {
        return Collections.emptyMap();
    }
    try {
        AgentComponentConfigDto modelComponentConfig =
            agentContext.getAgentConfig().getModelComponentConfig();
        // queryMemory needs bound model targetId; skip if not bound.
        if (modelComponentConfig == null || modelComponentConfig.getTargetId() == null) {
            return Collections.emptyMap();
        }
        boolean justKeywordMatch = resolveJustKeywordMatch(agentContext);
        // originalMessage is main query term; context empty, short-term history does not reverse-enhance this query.
        return conversationApplicationService.queryMemory(
            agentContext.getUser().getTenantId(), agentContext.getUser().getId(),
            agentContext.getAgentConfig().getId(), modelComponentConfig.getTargetId(),
            agentContext.getOriginalMessage(), "", justKeywordMatch,
            agentContext.isFilterSensitive());
    } catch (Exception e) {
        // Long-term memory is enhancement; query failure lets main dialogue continue.
        log.warn("查询长期记忆失败", e);
        return Collections.emptyMap();
    }
}, memoryLoadExecutor);

// Join point: main flow waits for both memory paths.
agentContext.setContextMessages(contextMessagesFuture.join());
Map<String, String> longMemoryMap = longMemoryFuture.join();

Design points: dedicated thread pool avoids common pool contention; long-term memory queries with empty context first. Boundaries and fallbacks: openLongMemory off → empty map; agent unbound to model → skip long-term retrieval; query exception logs warning and returns empty, main dialogue continues. Current long-term query uses originalMessage as primary retrieval term, independent of short-term memory reading; parallelism yields execution overlap, not "short-term history injected then enhanced query".

Semantic Retrieval Implementation

MemOS Search request parameters:

List<String> readableCubeIds = new ArrayList<>();
readableCubeIds.add("user_profile");
if (agentId != null) {
    readableCubeIds.add("agent_" + agentId);
}

MemOSClient.SearchRequest searchRequest = new MemOSClient.SearchRequest();
// One Search covers both user profile and current agent memory cubes.
searchRequest.setQuery(buildSearchQuery(userMessage, context));
searchRequest.setUserId(memOSUserId);
searchRequest.setTopK(DEFAULT_TOP_K);
searchRequest.setMode("fast");
searchRequest.setRelativity(0.45);
searchRequest.setDedup("mmr");
searchRequest.setReadableCubeIds(readableCubeIds);
searchRequest.setIncludePreference(true);
searchRequest.setPrefTopK(6);

List<MemOSMemory> searchResults =
    memOSClient.searchMemory(searchRequest).getMemories();
// Conversion filters low-score results, truncates single content, returns sorted by score descending.
result.addAll(convertToMemoryUnitDTOs(searchResults, tenantId, userId, agentId));
result.sort((a, b) -> {
    if (a.getScore() == null && b.getScore() == null) return 0;
    if (a.getScore() == null) return 1;
    if (b.getScore() == null) return -1;
    return Double.compare(b.getScore(), a.getScore());
});

readableCubeIds

includes user_profile and agent_{agentId}; preference retrieval enabled with prefTopK=6 to supplement user preferences. Results further filtered by score < 0.3 and truncated to 1000 characters per item to prevent low-relevance or overlong content from crowding context.

Token Budget Allocation Strategy

Following retrieval, userProfileRaw and agentMemoryRaw are scope-aggregated texts. To prevent memory from blowing up agent context, a token budget mechanism is used:

int budget = DEFAULT_LONG_MEMORY_TOKEN_BUDGET; // 4000 tokens
int userProfileRawTokens =
    TikTokensUtil.tikTokensCount(userProfileRaw != null ? userProfileRaw : "");

// user_profile max 60%; unused remainder given to agent memory.
if (userProfileRawTokens <= budget * 0.6) {
    userProfile = userProfileRaw;
    agentMemory = truncateLongMemory(agentMemoryRaw, budget - userProfileRawTokens);
} else {
    userProfile = truncateLongMemory(userProfileRaw, (int) (budget * 0.6));
    // Recalculate truncated tokens to avoid carrying pre-truncation estimate into remainder.
    int userProfileTokens =
        TikTokensUtil.tikTokensCount(userProfile != null ? userProfile : "");
    agentMemory = truncateLongMemory(agentMemoryRaw, budget - userProfileTokens);
}

Truncation algorithm uses line-by-line truncation to preserve complete semantics:

String[] lines = longMemory.split("
", -1);
for (String line : lines) {
    // Accumulate tokens per line, try to keep entire memory item intact.
    int lineTokens = TikTokensUtil.tikTokensCount(line + "
");
    if (currentTokens + lineTokens > budget) {
        break;
    }
    truncated.append(line);
    currentTokens += lineTokens;
}

User profile capped at 60% budget; if actual usage less, remainder allocated to agent memory. Both types accumulate tokens line-by-line, stopping when budget reached, avoiding splitting a single memory item.

Long-Term Memory Writing Implementation

Write Entry and Deduplication

At session end, system triggers memory write flow, first using Redis Set to record processed message hashes for deduplication.

Class notes: MemoryRpcServiceImpl orchestrates post-session writes; onSessionEndAsync acquires session-level Redis lock, then processSessionEnd filters new content by message hash and pre-records processed hashes. Based on agent config, selects provider: MemOS path runs LLM memory judgment first, then MemoryPersistenceServiceImpl persists; MySQL/Mem0 paths assemble full context and latest user input then call createMemory. After memory processing, same async thread checks if context summary generation needed.

Design points:

Distributed lock prevents concurrent processing of same session.

MD5(role:content) generates unique message identifier.

TTL 7 days auto-cleans expired records. onSessionEndAsync uses 10-minute session lock; processed message hash TTL 7 days. Hash written before LLM judgment and MemOS persistence to block duplicate end events. This layer is best-effort: Redis read failure treats entire session as new content; hash write failure only logs. No automatic replay on external service failure, so operations must monitor both duplicate writes and incomplete processing.

LLM Intelligent Judgment

Full prompt defines memory categories, scope, importance, and JSON output constraints; below retains only new-message marking and model call main line. judge calls LLM first; on exception or empty return, falls back to rule-based judgment using only new messages. When LLM returns non-empty, missing fields are defaulted, empty content filtered, and each memory truncated to 200 characters.

private MemoryJudgeResult judgeByLLM(Long tenantId, Long modelId,
    List<MemoryMessage> fullContext, List<MemoryMessage> newMessages) {
    Set<String> newMessageKeys = newMessages.stream()
        .map(m -> (m.getRole() != null ? m.getRole() : "user")
            + ":" + (m.getContent() != null ? m.getContent() : ""))
        .collect(Collectors.toSet());

    StringBuilder content = new StringBuilder("## 完整对话上下文

");
    for (MemoryMessage message : fullContext) {
        String role = message.getRole() != null ? message.getRole() : "user";
        String text = message.getContent() != null ? message.getContent() : "";
        content.append(newMessageKeys.contains(role + ":" + text)
            ? "[[NEW] " : "[")
            .append(role).append("]: ").append(text).append("
");
    }

    MemoryJudgeResult result = iModelRpcService.call(
        tenantId, modelId, JUDGE_SYSTEM_PROMPT, content.toString(),
        new ParameterizedTypeReference<MemoryJudgeResult>() {});
    return result == null ? null : validateAndFixResult(result);
}

Judgment points:

Generates message keys by role:content and marks focus content; historical messages with same content may also be hit.

Defines importance ranges (1-10) for 5 information categories.

Explicit scope attribution rules.

Deduplication and Conflict Handling

Local similarity deduplication before persistence reduces unnecessary writes:

Class notes: MemoryPersistenceServiceImpl#calculateSimilarity judges sequentially by exact match, contains, character-level Jaccard (threshold 0.7), and short-text Levenshtein (threshold 0.8); similar marks as duplicate; null or below threshold returns none. This only handles intra-batch local pre-deduplication; cross-history memory conflicts delegated to subsequent conflict service. batchPersist completes local deduplication, groups by memCubeId, then calls batchDetectConflict to get conflict results per candidate memory. Core loop decides per item: write new memory, skip, or record old memory for invalidation:

for (int i = 0; i < memoryList.size(); i++) {
    MemoryToSave memory = memoryList.get(i);
    MemoryConflictResult conflict = i < conflictResults.size()
        ? conflictResults.get(i) : MemoryConflictResult.noConflict();

    if (!conflict.isHasConflict()) {
        messagesToStore.add(createMessage(memory.getContent()));
        continue;
    }

    ConflictResolutionResult resolution =
        memoryConflictService.resolveConflict(
            tenantId, memOSUserId, memCubeId,
            memory.getContent(), conflict, modelId);
    if (resolution.isShouldStoreNew()) {
        messagesToStore.add(createMessage(memory.getContent()));
        if (CollectionUtils.isNotEmpty(resolution.getMemoryIdsToExpire())) {
            memoryIdsToDelete.addAll(resolution.getMemoryIdsToExpire());
        }
    }
}

"Batch" here means grouping by memCubeId and reusing a batch of pending writes; conflict service's batch method internally still calls detection API per item, then decides per item to keep new, skip, or mark old. Thus it reduces grouping and orchestration overhead but is not equivalent to a single network request completing all conflict judgments.

Write-Before-Delete Strategy

To avoid data loss, adopts "write new memory first, then delete old memory" strategy.

Class notes: MemoryPersistenceServiceImpl#batchPersist assembles MemOS AddMemoryRequest per memCubeId, writes userId, writableCubeIds, and messages, calling addMemory in async and fine modes. Only when response contains new memories are conflict old memory IDs passed to deleteMemories; delete exceptions only log warning, newly written memories retained.

This is a "write-priority" risk control, not transactional consistency: MemOS is external HTTP service, platform cannot wrap both insert and delete in a local transaction. Current delete failures are best-effort warnings; old memories may linger temporarily, cleaned up later by subsequent retrieval or next conflict processing.

Core Design and Operational Observability

This implementation splits memory processing into two independent chains: pre-request parallel loading of short-term history and long-term memory; post-session async completion of filtering, deduplication, and persistence. Core strategies concentrate on four points:

Parallel Loading: Short-term and long-term memory submitted to same dedicated thread pool; main flow joins at merge point.

New Message Marking: Uses [[NEW] to tag current new messages, letting LLM extract new information with full context.

Budget Control: Short-term history uses sliding window with reserved summary space; long-term memory allocates fixed tokens after scope grouping.

Reliable Writing: Redis lock and hash provide idempotency; MemOS side writes new memories first, then best-effort deletes conflicting old memories.

Operational Observability: System provides runtime overview, memory type distribution, time trends, top agents, and cube details for understanding memory data scale, classification, sources, and change trends. Supports viewing long-term memory search trends by time range, suitable for daily inspection, data distribution analysis, and anomaly localization.

Summary and Future Work

Short-term memory brings current session into next turn; long-term memory brings filtered user facts and agent experience into future sessions. Both chains read in parallel before request, linked after session via async tasks; long-term memory further controls injected content by scope and token budget.

Current solution covers Redis/MySQL fallback, MemOS semantic retrieval, LLM judgment, local deduplication, conflict handling, and write-before-delete. Still need to supplement latency and failure rate observability under production traffic, and compensation mechanisms for external memory service failures.