Hermes Prompt Runtime: Managing Provider, Prompt, Memory, and Context

Provider Resolution – First Gate of Prompt Runtime

Before any model call, the Agent determines which Provider to use. Providers include Anthropic Messages API, OpenAI‑compatible endpoints, OpenRouter, AI Gateway, Codex Responses API, or custom enterprise endpoints. The selected Provider defines the API mode, tool‑call format, caching ability, credential source, retry and fallback logic.

Provider Resolution output provider : the service or compatible channel for the current round. api_mode : how subsequent messages are translated into the Provider’s request format. base_url & api_key : endpoint and credential source, establishing permission boundaries. fallback information : whether a backup model can be switched to on failure and whether caching capabilities need re‑evaluation.

The resolver follows a priority order: explicit runtime request → config.yaml model and Provider settings → environment variables → Provider defaults. This order prevents stale environment variables from overriding saved model choices, ensuring each call has an explainable and stable source.

Prompt Builder – Constructs a Stable System Prefix

The prompt_builder.py module assembles system‑level information in a deterministic chain:

Agent Identity (e.g., SOUL.md or default identity)

Tool Guidance (usage rules, memory‑write rules, history‑retrieval hints)

Optional system messages

Frozen MEMORY snapshot ( MEMORY.md)

Frozen USER snapshot ( USER.md)

Skills Index (summaries of available skills)

Project Context (e.g., HERMES.md, AGENTS.md, CLAUDE.md, Cursor rules)

Timestamps

Platform hints

This ordering solves two problems:

Semantic continuity – every model turn sees the same identity, memory, and behavior constraints, avoiding ad‑hoc patches.

Prompt caching – a consistent prefix increases the likelihood of cache hits on the Provider side.

Injecting temporary recalls, gateway overlays, or tool fragments into the system prompt each turn would jitter the prefix, breaking both cost optimization and semantic stability.

Frozen Memory Snapshots – Isolate Mid‑Session Writes

At session start, the contents of MEMORY.md and USER.md are injected as snapshots into the system prompt. Writes to the memory tool during the session update the on‑disk files but do not immediately alter the already‑built system prompt.

Engineering problems solved by frozen snapshots Semantic consistency : system prompts are not repeatedly overwritten by mid‑session memory writes. Cache stability : the stable prefix remains cache‑friendly. Debug clarity : when inspecting a model output, the memory snapshot used is identifiable.

Consequently, the current round sees only the initial stable memory; later writes become visible only in the next session or after a forced rebuild. This mirrors transaction isolation in databases: writes can occur, but the running transaction’s view of the system state remains unchanged.

API‑Call‑Level Temporary Additions – Separate Path

Hermes reserves an API‑call‑only layer for transient data such as ephemeral_system_prompt, prefill messages, gateway‑derived session‑context overlays, and turn‑level recalls injected into the user message. These items answer “what does this call need to know?” without altering the long‑term system prompt.

For multi‑entry Agents (Telegram, Discord, API Server, ACP), each entry may need to add temporary context, source identifiers, platform limits, or external recalls. Placing these in the temporary layer prevents entry‑specific noise from polluting the stable system prefix.

Context Compressor – Handles History Pressure

Long conversations eventually hit context limits. Hermes employs a two‑tier compression system:

Gateway session hygiene layer that prevents unbounded accumulation before the Agent.

Internal ContextCompressor that operates based on actual API token usage.

The default compressor first removes large tool‑output blocks, then protects the head and tail of the history to keep tool calls and their results grouped. The middle portion is summarized by an auxiliary model into a structured abstract that retains:

Current task goals and user constraints. Completed progress, in‑flight nodes, and blocking points. Key decisions and their justification. Paths of files that have been read, modified, or created. Next actions and any critical errors, configurations, or command results that must be kept.

After compression, the stable system prompt remains largely untouched, allowing Prompt Caching to stay effective while the compressed history continues the session.

Engineering Takeaways

Hermes Prompt Runtime demonstrates that a stateful Agent must explicitly define the lifecycle of each information class. A more stable system prompt, restrained temporary context, and clear compression boundaries make the Agent behave like a maintainable long‑running engineering system.