Bridging the Gap: Enforcing Discipline in AI Agents for Reliable Performance

Introduction

Large language models (LLMs) can generate impressive one‑shot responses, but when an AI agent is required to operate over multiple turns with precise memory and state management, serious gaps appear. Issues like catastrophic context drift, knowledge overtake, and unreliable state handling expose the limits of pure prompt engineering.

Key Problems

Crisis Emergence : A travel‑planning agent loses its original goal after a single interaction, confusing "Shanghai" with "Beijing" and hijacking the task context.

Wrong Path : Overly complex prompts are used to trick the LLM, leading to inconsistent behavior and instruction drift.

Dead‑End Scenarios : The agent either floods the UI with repeated waiting messages or terminates abruptly, breaking the conversation flow.

Design Philosophy

To overcome these issues, the team introduced a tightly coupled architecture where deterministic code handles state while the LLM provides reasoning. The core concepts are:

Code Lock : Java code with Redis stores a reliable, deterministic state machine ("long‑term memory").

Attention Anchor : A short‑lived todo.md file acts as a "sticky note" to keep the LLM focused on the current sub‑task.

Implementation Details

Each conversation thread has a JSON object in Redis containing todo_content (the latest snapshot of todo.md) and waiting_for (the pending question). The workflow is:

When the agent needs to ask the user, it writes an ask_lock.json file and emits an <ask> signal, then terminates.

On user reply, the handler checks the lock, validates the answer, updates the Redis state, deletes the lock, and resumes processing.

Key Java snippets:

// A2ARequestHandler.java
function processRequest(userInput, threadId) {
    // 1. Load context from Redis
    String contextJson = redis.get("context:" + threadId);
    String finalPrompt;
    if (contextJson != null) {
        JSONObj context = parseJson(contextJson);
        if (context.has("waiting_for")) {
            finalPrompt = "Context: you previously asked '" + context.get("waiting_for") + "'. "
                + "Current plan: " + context.get("todo_content") + ". "
                + "User answer: '" + userInput + "'. "
                + "Your task: update the plan strictly according to the answer and decide the next step.";
            // Clear waiting state
            context.remove("waiting_for");
            redis.set("context:" + threadId, toJson(context));
        } else {
            // ... other logic ...
        }
    } else {
        finalPrompt = userInput;
    }
    // Run the agent with the constructed prompt
    agentManager.run(finalPrompt);
}

// EventProcessingService.java
function processEvent(event, threadId) {
    // 1. Detect file rewrite of todo.md
    if (event.isToolCall("full_file_rewrite") && event.getParam("file_path") == "todo.md") {
        String newTodo = event.getParam("file_contents");
        JSONObject ctx = parseJson(redis.get("context:" + threadId));
        ctx.put("todo_content", newTodo);
        redis.set("context:" + threadId, toJson(ctx));
    }
    // 2. Detect ask request
    if (event.isToolCall("ask")) {
        String question = event.getParam("text");
        JSONObject ctx = parseJson(redis.get("context:" + threadId));
        ctx.put("waiting_for", question);
        redis.set("context:" + threadId, toJson(ctx));
        emitEndStreamSignal();
    }
    return event;
}

Outcome

By delegating all cross‑turn state to code and persisting it in Redis, the agent regains deterministic behavior. The attention anchor ( todo.md) prevents context hijacking, while strict prompt rules (e.g., absolute commands, no internal knowledge usage) keep the LLM disciplined. The result is a reliable, production‑ready AI agent that can handle multi‑step planning without drifting.

Conclusion

The journey demonstrates that a robust AI agent requires both code for deterministic control and carefully crafted prompts for intelligent reasoning. Treating code, prompts, and file‑based state as a single, inseparable system is essential for moving from experimental prototypes to industrial‑grade deployments.