Deep Dive into Hermes Agent: The Memory Architecture That Makes AI Smarter

1. Introduction

From 2024‑2025 many AI‑agent frameworks suffered from memory hallucination : they attached a vector database and claimed “long‑term memory”, yet each new session required the user to repeat the same codebase, project context and preferences. Hermes Agent was released on 2026‑02‑25 by Nous Research and quickly became the fastest‑growing open‑source agent framework, surpassing 105 000 GitHub stars.

2. Project Lineage

Hermes 3 (2024‑08) – built on Meta Llama 3.1 (8B/70B/405B). Introduced robust tool‑call JSON tags and improved multi‑turn coherence (arXiv:2408.11857).

Hermes 4 (2025) – added mixed‑precision inference, 14B/70B/405B FP8 variants and large‑scale synthetic data.

Hermes Agent (2026‑02) – combines persistent memory, automatic skill creation and multi‑platform support. The model component is interchangeable (supports 200+ models).

Key release data (v0.7.0, 2026‑04‑03): 10 700 lines in run_agent.py, 10 000 lines in cli.py, MIT license, Linux/macOS/WSL2 support, 47 built‑in tools (19 toolsets), 15+ messaging platforms, 18+ LLM providers.

3. System Architecture

The system is divided into four major subsystems.

Core Subsystems AIAgent Loop – orchestrated by run_agent.py. Prompt System – builds system prompts ( prompt_builder.py, prompt_caching.py, context_compressor.py). Provider Resolution – maps (provider, model) to runtime credentials. Tool System – central registry ( tools/registry.py) with 47 tools across 19 toolsets.

Infrastructure Subsystems

Session Persistence (SQLite with FTS5).

Messaging Gateway (18 adapters for Telegram, Discord, Slack, Signal, Matrix, Mattermost, Email, SMS, DingTalk, Feishu, WeCom, WeChat, BlueBubbles, QQBot, HomeAssistant, Webhook, API Server).

Plugin System (discoverable via ~/.hermes/plugins/, project‑level .hermes/plugins/, and pip entry points).

Cron Scheduler (native, not shell‑based).

Advanced Subsystems

ACP Integration – exposes Hermes as a native IDE agent via stdio/JSON‑RPC.

RL/Environments/Trajectories – Atropos reinforcement‑learning integration.

4. Core Engine: AIAgent Loop

run_conversation()
  1. Generate task_id if missing
  2. Append user message to history
  3. Build or reuse cached system prompt
  4. Apply pre‑compression if context >50%
  5. Construct API request (chat_completions / codex_responses / anthropic_messages)
  6. Inject temporary prompts (budget warnings, context pressure)
  7. Apply Anthropic prompt‑cache flag when needed
  8. Issue interruptible API call
  9. Process response:
     - tool_calls → execute tools, append results, repeat step 5
     - text → persist session, optionally refresh memory, return

Message role alternation is strict: system → user → assistant → user → … for normal flow and assistant (tool_calls) → tool → … → assistant during tool execution. Consecutive assistant or user messages are prohibited; only consecutive tool messages are allowed.

5. API Modes

chat_completions

– OpenAI‑compatible endpoints (OpenRouter, custom, most providers). codex_responses – OpenAI Codex / Responses API. anthropic_messages – native Anthropic Messages API.

Resolution priority: explicit api_mode argument → provider‑specific detection → base‑URL heuristic → default chat_completions.

6. Turn Lifecycle

run_conversation()
  1. Generate task_id (if missing)
  2. Append user message
  3. Build or reuse cached system prompt
  4. Pre‑compression check (>50% of model window)
  5. Build API payload (model‑specific format)
  6. Inject temporary prompts (budget, context pressure)
  7. Apply Anthropic cache flag
  8. Call _api_call_with_interrupt() (background thread)
  9. Handle response:
     - tool_calls → execute, append result, repeat step 5
     - text → persist session, refresh memory, return

API calls run in a background thread; an interrupt event (new user input or /stop) aborts the request, discards partial output and allows the agent to react cleanly.

7. Memory Architecture (Layered)

Layer 1 – Core Persistent Memory MEMORY.md (≈2 200 chars) – environment facts, project conventions, discovered tool issues, completed tasks, effective skills. USER.md (≈1 375 chars) – user profile (name, role, timezone, communication preferences, technical level).

Both files are injected as a frozen snapshot at session start and never change during the session. Updates are written to disk immediately but become visible only in the next session. Capacity management: when usage exceeds 80 % of the character limit, the agent must merge entries before adding new ones.

Layer 2 – Procedural (Skill) Memory – skills are stored as SKILL.md files (agentskills.io schema). The agent automatically creates a skill after a complex task, enabling true “learning from experience”.

Layer 3 – Session Search – all CLI and message sessions are stored in SQLite ( ~/.hermes/state.db) with FTS5 full‑text search, allowing on‑demand retrieval of past conversations.

Layer 4 – External Memory Providers (available from v0.7.0). Eight plug‑in providers (Honcho, Mem0, OpenViking, Hindsight, Holographic, RetainDB, ByteRover, Supermemory) can be selected at runtime and run in parallel with the built‑in memory.

8. Skill System (Procedural Memory)

Each skill is a Markdown file with a header block defining name, description, version, platforms, and metadata. Example header:

---
name: my-skill
description: Brief description of the skill
version: 1.0.0
platforms: [macos, linux]
metadata:
  hermes:
    tags: [python, automation]
    category: devops
    fallback_for_toolsets: [web]
    requires_toolsets: [terminal]
    config:
      - key: my.setting
        description: What this setting controls
        default: "value"
        prompt: "Set the value"
---
# Skill Title
## When to Use
... (description) ...
## Flow
1. First step
2. Second step
## Common Errors
- Known failure mode and fix
## Validation
How to confirm success

Skill management is performed via the skill_manage tool with operations create, patch, edit, delete, write_file, and remove_file. patch is preferred because it transmits only the changed text, saving tokens.

Skills can be discovered from multiple hubs: official – built‑in repository skills. skills‑sh – Vercel public skill directory. well‑known – URL‑based discovery. github, url, clawhub, lobehub – third‑party marketplaces.

Hub commands include hermes skills browse, hermes skills search, hermes skills install, hermes skills audit, and hermes skills publish. All installed skills undergo a security scan for prompt‑injection, credential leakage and dangerous commands.

9. Prompt Assembly System

The system prompt is built from ten ordered layers, each optionally cached. Layers are:

Agent identity ( SOUL.md or default identity).

Tool behavior guidelines (memory usage, session search, tool execution).

Honcho static block (dialectic user modeling).

Optional system messages from config or API.

Frozen MEMORY.md snapshot.

Frozen USER.md snapshot.

Compressed skill index (list of installed skills).

Context files ( .hermes.md, AGENTS.md, .cursorrules, etc.).

Timestamp and session ID.

Platform‑specific prompt (CLI, Telegram, Discord, …).

Each file is security‑scanned for prompt‑injection patterns, stripped of YAML front‑matter, and truncated to a 20 000‑character limit using a 70/20 head‑tail strategy.

During an API call, temporary layers (e.g., budget warnings, Honcho recall, gateway‑specific context) are injected but never become part of the cached system prompt, preserving prompt stability for caching.

10. Tool Registry and Execution

All tools live under tools/ and self‑register with tools/registry.py at import time, eliminating manual registration lists. Execution modes:

Sequential – single tool call runs in the main thread.

Concurrent – multiple tool calls are dispatched via ThreadPoolExecutor.

Interactive exception – tools marked clarify are forced to run sequentially.

Tool categories (47 tools, 19 toolsets) include terminal & execution, file system, web/browser automation, memory & session search, skill management, multimedia, and MCP integration.

Special tool execute_code enables “Programmatic Tool Calling”: the agent writes a complete script and executes it in a single LLM call, dramatically reducing token cost compared to step‑by‑step tool loops.

Delegation via delegate_task spawns independent sub‑agents with their own context, terminal and budget (default 50 iterations). The parent aggregates results, allowing true parallel workflows.

11. Message Gateway Architecture

The gateway ( gateway/run.py, ~9 000 lines) runs as a long‑living process and routes messages from 18 adapters (Telegram, Discord, Slack, WhatsApp, Signal, Matrix, Mattermost, Email, SMS, DingTalk, Feishu, WeCom, WeChat, BlueBubbles, QQBot, HomeAssistant, Webhook, API Server).

Processing flow (simplified):

Platform event → Adapter → MessageEvent
  → GatewayRunner._handle_message()
    1. Authorization check (global allow‑all, per‑platform allowlist, DM pairing, etc.)
    2. Parse session key (isolated per platform)
    3. Slash‑command handling (/model, /skills, /compress, …)
    4. Create AIAgent with session history
    5. AIAgent.run_conversation() (progress updates streamed back)
    6. Adapter sends final response to user

DM pairing provides a code‑based onboarding flow: an unknown user receives an 8‑character pairing code, the owner approves it via hermes pairing approve <platform> <code>, after which the user is permanently authorized. Security features include per‑platform session isolation, atomic SQLite writes, and rate‑limited pairing attempts.

12. Context Compression and Caching

Compression is triggered when the context exceeds thresholds:

CLI pre‑compression: >50 % of model window.

Gateway auto‑compression: >85 % of model window.

Compression workflow:

Flush memory to disk.

Summarize intermediate turns with an auxiliary LLM.

Preserve the last N messages (default 20) and all tool call/result pairs.

Create a new child session ID to keep the full lineage.

From v0.7.0 the context engine is pluggable via agent/context_engine.py. The default engine ( context_compressor.py) uses lossy summarization; custom engines can be supplied by implementing should_compress and compress.

13. Terminal Backends and Deployment

Six terminal backends are supported, each offering a different isolation level:

local – no isolation (development, trusted users).

ssh – remote machine, dangerous‑command check enabled.

docker – container isolation; dangerous‑command check is skipped because the container is the boundary.

singularity – HPC container isolation.

modal – cloud sandbox.

daytona – cloud sandbox with persistent workspace.

Docker backends apply a hardened security profile ( _SECURITY_ARGS) that drops all Linux capabilities except a minimal whitelist (e.g., DAC_OVERRIDE, CHOWN, FOWNER), disables new privileges, limits PIDs, and mounts limited /tmp, /var/tmp and /run as tmpfs.

Resource limits (CPU, memory, disk) are configurable in ~/.hermes/config.yaml.

14. Security Model: Defense‑in‑Depth

Hermes implements a seven‑layer security architecture:

User authorization (allowlists, DM pairing).

Dangerous‑command approval (manual, smart, or off).

Container isolation (Docker, Singularity, Modal, Daytona).

MCP credential filtering (environment variable isolation).

Context file scanning (prompt‑injection, invisible Unicode, credential patterns).

Cross‑session isolation (separate SQLite stores per platform/user).

Input sanitization for terminal backends (path validation, argument checks).

Dangerous‑command detection ( tools/approval.py) covers >30 patterns, including recursive rm -r /, chmod 777, chown -R root, database DROP TABLE, systemctl actions, and shell pipelines like curl … | sh. Approval modes:

manual (default) – always ask the user.

smart – LLM evaluates risk; low‑risk commands auto‑approved.

off – disables checks (intended for trusted CI/CD environments).

When a terminal backend provides its own isolation (Docker, Singularity, Modal, Daytona), the dangerous‑command check is skipped because the container itself is the security boundary.

Memory entries undergo the same security scan for prompt‑injection directives, credential‑theft patterns, SSH backdoors, and invisible Unicode. Duplicate entries are rejected.

15. Reinforcement Learning and Trajectory Generation

Hermes integrates the Atropos RL environment from Nous Research. The batch_runner.py tool allows users to configure workers, batch size, and toolset distributions to generate large numbers of tool‑call trajectories. Generated trajectories are stored in SQLite ( ~/.hermes/state.db) and can be exported as ShareGPT, OpenAI message format, or custom JSON for fine‑tuning.

16. Design Principles and Engineering Philosophy

Prompt stability – system prompts never change mid‑conversation (except explicit /model switches), maximizing cache hits.

Observability – every tool call is visible to the user (CLI spinners, gateway chat messages).

Interruptibility – API calls and tool executions can be cancelled by new input or /stop.

Platform‑agnostic core – a single AIAgent class serves CLI, gateway, ACP, batch, and API server.

Loose coupling – optional subsystems (MCP, plugins, memory providers, RL) are discovered via registries, not hard‑coded.

Profile isolation – each hermes -p <name> profile has its own HERMES_HOME, config, memory, session store and gateway PID, enabling concurrent independent agents.

17. Comparison with OpenClaw

Core Architecture – Hermes treats the learning loop as first‑class; OpenClaw is tool‑call centric.

Memory System – Hermes uses layered hot/warm/cold memory with active curation; OpenClaw relies mainly on vector retrieval.

Skill System – Hermes auto‑generates and continuously improves skills; OpenClaw uses static skill definitions.

Learning Mechanism – Hermes extracts persistent knowledge and employs Honcho user modeling; OpenClaw has no built‑in learning.

Terminal Backends – Hermes supports six backends (including cloud sandboxes); OpenClaw offers limited options.

Signal Support – Hermes includes Signal; OpenClaw does not.

Security Model – Hermes implements a seven‑layer defense‑in‑depth model; OpenClaw has fewer layers.

Model Flexibility – Hermes works with 200+ models from 18+ providers; OpenClaw is more restricted.

RL Integration – Hermes includes Atropos; OpenClaw has none.

License – Hermes is MIT; OpenClaw’s license varies.

18. Known Limitations and Ongoing Security Research

Memory poisoning risk – skills are stored as plain Markdown without cryptographic signatures. A malicious injection during a session can be persisted as a skill and later executed without verification. Current mitigations are prompt‑injection scanning and manual audits; future work includes signed skill files.

Memory capacity – MEMORY.md (2 200 chars) and USER.md (1 375 chars) limit the amount of active knowledge that can be injected into the prompt. External memory providers are required for larger knowledge bases.

Synchronous engine – the core AIAgent loop is synchronous; high‑throughput batch workloads rely on the separate batch_runner.py multiprocess architecture.

19. Conclusion

Hermes Agent represents a paradigm shift in autonomous AI‑agent engineering by treating the learning loop as a first‑class architectural element. Its layered memory system, automatically generated and verifiable skills, robust seven‑layer security model, and extensive platform support provide a solid foundation for building production‑grade, continuously improving AI assistants. Ongoing work on skill signing, richer external memory backends, and deeper RL integration will further strengthen its position as a leading open‑source framework for autonomous agents.