From Chatbot to AI Agent: Why a Single While Loop Fails in Production

1. Starting from an Operations Problem: Defining What We Can Prove

The demo scenario: payment-service error rate rose over the last 30 minutes; the agent must analyze possible causes and suggest a rollback if warranted. The full platform will later correlate metrics, logs, and release records, then execute changes after approval. Week 1 (W01) implements only the shortest slice: create a task, query simulated metrics, and produce a result with cited evidence.

A mock tool returns a fixed error_rate=0.08 with synthetic=true, a test fixture not a live measurement. The mock provider's final answer explicitly states that metrics alone cannot determine root cause — logs and release records are needed — and no rollback is executed. This restraint matters: tool evidence proves an observation exists, not causality. Evidence and causal judgment are separate layers. W01 validates that cited evidence IDs actually exist; causal reasoning quality is left for later datasets and human evaluation.

Acceptance is split: (1) does the task terminate correctly? (2) does the result reference only evidence actually obtained in this task? Model diagnostic adequacy is not judged by a single polished demo.

2. Choosing an Engineering Boundary: Why an In-Process Background Execution Module

Four options were compared:

Synchronous HTTP handler loop — lowest cost, but request lifecycle binds long tasks; query and cancel hard to separate. Not adopted.

One goroutine per request — low cost, but no queue bound; concurrency and shutdown management scatter. Not adopted.

Finite queue + background consumer — moderate cost; still volatile but responsibilities and resource boundaries are clear. W01 implementation.

Independent worker + PostgreSQL — higher cost; needs persistence, claim, lease, transaction design. Planned for W04.

ADR-0007 selects a single cmd/api process combining the HTTP module, in-memory repository, and a background execution module. The HTTP handler never calls the agent loop; it only submits a task ID to the scheduler. The scheduler runs one consumer with a pending-queue capacity of 32. Process boundaries are merged for now; execution responsibility stays separate. W04 will extract an independent worker once a shared database exists.

Current component graph (all nodes map to this week's code): Local Client → Gin / Demo Auth (create, query, cancel) → In-Memory Repository → Finite Queue → Background Runtime → Mock Provider → Whitelisted Mock Tools.

The queue is not the source of truth; the repository holds queryable in-process state. Neither survives a restart. Max stored tasks = 1000; capacity full returns 503; no automatic cleanup. This bound prevents unbounded growth in the prototype but is not a mature retention policy.

3. Breaking the Loop into Three Replaceable Contracts

Key interfaces (in internal/, not a published SDK):

// internal/llm/provider.go
type LLMProvider interface {
    Complete(context.Context, CompletionRequest) (CompletionResponse, error)
}

// internal/tool/executor.go
type ToolExecutor interface {
    Execute(context.Context, domain.Principal, llm.ToolCall) (domain.Evidence, error)
}

// internal/agent/runtime.go
type AgentRuntime interface {
    Run(context.Context, string) error
}

Provider

decides the next suggested action; ToolExecutor validates principal, tool name, and arguments then executes; Runtime controls the loop, budgets, and state commits. Repository provides Create, Get, Claim, Append, Finish, Cancel with concurrent state checks inside lock-held critical sections.

A provider response must contain exactly one of ToolCall or Final — both present or both absent terminates as INVALID_DECISION. Runtime never guesses from ambiguous model output, nor treats "model says success" as tool success.

Normal execution is two steps: (1) model requests a metric query, tool returns Evidence; (2) model returns Final. Final must include evidence_ids that all exist in the task's evidence set, and both summary and reference set must be non-empty. Only then does the task enter SUCCEEDED.

Tool output remains low-trust data. The mock returns structured fixed values; future log/document integration must not elevate tool text to system instructions. W01 verified reference integrity; it does not claim to have solved prompt injection or factual correctness.

4. State Machine: Cancellation Is Not a Boolean Flag

W01 uses a subset of the full architecture's states: PENDING, RUNNING, SUCCEEDED, FAILED, CANCELLED. No READY_TO_RESUME or approval/retry states yet.

Transitions are enforced in code, not just a diagram: Memory.Claim confirms PENDING inside a lock before entering RUNNING; Finish calls workflow.Allowed inside the lock; Append only permits RUNNING state. Terminal states have no legal outgoing transitions.

Cancellation first flips the task to CANCELLED in the repository, then notifies the worker's cancel function. Even if the old loop later receives a tool result and attempts Append or Finish, the repository rejects it. This is more reliable than a cancelled bool inside the loop because cancellation and result writes have an explicit concurrency ordering point.

If success acquires the lock first, cancel returns the already-succeeded task; if cancel acquires the lock first, the success write fails. The system cannot retroactively undo a success that already happened, nor let a stale result overwrite a cancellation — both behaviors must be explainable to the caller.

The repository returns deep-copied snapshots to prevent callers from mutating internal Steps, Evidence, or Output references. Concurrency safety requires not only locking the map but also ensuring no mutable aliases leak after the lock releases.

This is still single-process terminal-state protection. No lease epochs, no external tool fencing. If an external system has already accepted a change, local cancellation cannot auto-revoke it; idempotency, state query, or manual verification are needed. This boundary was made explicit in week one.

5. Finite Execution: Every Stop Condition Needs a Reason

Default Limits: max 6 agent steps, max 4 tool calls, total task duration 5 seconds (from claim, excluding queue), per-model/tool call timeout 1 second. Each step allows at most one model request, so total model requests are also bounded by max steps.

Step count includes the final answer step. If the sixth tool step completes and the provider still hasn't returned Final, runtime ends with MAX_STEPS_EXHAUSTED. No extra step is secretly run to wait for an answer.

Repeated calls are detected by a key of tool name + normalized JSON arguments (parse then re-serialize to avoid field-order differences). Second occurrence returns REPEATED_TOOL_CALL without re-executing the tool. This suits W01's read-only diagnostic demo but is not a general idempotency mechanism: the same query may need fresh results later, and JSON normalization (e.g., 1 vs 1.0) ≠ full business semantic deduplication. Future work must refine per tool type, step intent, and retry rules; it cannot replace idempotency keys for external writes.

Timeouts split into task total ( TASK_TIMEOUT), model single-call ( LLM_TIMEOUT), tool single-call ( TOOL_TIMEOUT). Total-task context is checked first to avoid misreporting total timeout as a single-call timeout.

Go context provides cancellation signals; callees must cooperate. The mock checks ctx.Err() or waits on ctx.Done(), so it exits on budget. Swapping in an SDK that ignores context means the timeout signal cannot forcibly kill the call. The implementation does not spawn a leaked goroutine to fake timely returns. Real adapter cancellation semantics must be re-verified at integration time (see official context docs at https://pkg.go.dev/context).

Token estimation, precise usage metering, and per-task cost quotas are not implemented. The article does not equate "has max steps" with "full cost governance."

6. Model Proposes, Server Authorizes

Demo Auth activates only with AEGIS_DEMO_AUTH=true and non-empty AEGIS_DEMO_TOKEN; HTTP listens on 127.0.0.1. Bearer token validation creates a fixed principal demo/developer. Request body cannot contain tenant_id — strict JSON decoding rejects it. Repository.Get and Cancel check tenant and principal; mismatch or missing returns 404. Multi-tenant end-to-end OIDC mapping is not implemented; repository tests separately verify cross-tenant and cross-principal rejection. ToolExecutor further validates principal and the query_metrics whitelist: resource limited to payment-service, time window to 30m. Illegal fields, extra JSON, wrong time window, non-whitelisted resources are all rejected. Dynamic resources allowed by the step-exhaustion fixture are purely local synthetic data; no cluster connection exists.

When the mock provider emits a rollback suggestion, runtime still routes it through the same ToolExecutor, resulting in TOOL_DENIED. Model output is not an authorization credential. This is a baseline security boundary already in code; full RBAC, resource-level policies, approval binding, and re-authorization are slated for W06.

Create endpoint limits body to 8192 bytes, prompt to 1000 Unicode code points, rejects unknown fields and multiple JSON documents. Errors return stable public codes; underlying provider or tool exception details are never echoed directly, preventing future adapters from leaking addresses, credentials, or sensitive text. W01 stores no real secrets or raw production logs, and has no persistent audit or full redaction pipeline.

7. Making Failure an Experiment, Not Imagination

Test environment: Linux amd64, Go 1.26.2, Gin v1.10.0, CGO_ENABLED=1, GCC 13.3.0; dependencies locked via go.mod / go.sum. 9 logical CPUs, ~9.7 GiB RAM — visibility ≠ exclusive quota, so no performance conclusions are drawn.

Code passes go test -race -count=1 ./... and go vet ./..., plus a built binary running HTTP acceptance. Race detector only covers executed paths; it cannot prove absence of races under arbitrary scheduling (see official race detector docs at https://golang.org/doc/articles/race_detector.html).

Twelve experiments (table below) cover normal diagnosis, simulated tool error, repeated tool suggestion, max steps exhausted, tool budget exhausted, model/tool/task timeouts, user cancellation, rollback suggestion, non-existent evidence ID, 20 concurrent claimants, and restart query of old tasks. All pass. The HTTP script covers 9 task scenarios and 2 request-rejection checks; it is serial functional acceptance, not a load test. Runtime failure matrix separately covers total timeout, max steps, and invalid provider decisions. All results and commands saved in experiments.md and raw/; no P95, QPS, or inference cost statistics.

Package statement coverage: Agent 88.0%, API 88.1%, Repository 81.2%, scheduler & workflow 100.0%, tools 57.1%; mock provider and domain have no standalone package coverage but are exercised by other packages. Whole-repo statement coverage via go tool cover -func is 75.5%. These numbers are not rounded to "80%," and 100% coverage is not equated with logical correctness.

Failure path diagram illustrates a key distinction: after stopping, the reason remains queryable, not just an error in logs.

Model suggests tool call
→ Check repeat & call budget
→ If reject/error/timeout → FAILED + explicit error code
→ If allow → Tool independent auth & execute
→ Append Step / Evidence
→ Next step

A development-time ConnectionRefused occurred because server and client ran in isolated network namespaces — two 127.0.0.1 endpoints differed. Fix: local_experiments.py starts server, runs acceptance, stops server in the same environment. The prototype's listen address was not changed to a public interface to bypass the issue. Regular readers on one machine with two terminals can still follow the README.

8. Reproduction: From Task Creation to Final Result

Unzip the project, enter aegis-agent, use Go 1.26.2 with a C compiler:

go mod download
go test -race -count=1 ./...
go vet ./...
go build -o bin/aegis-api ./cmd/api
AEGIS_DEMO_AUTH=true AEGIS_DEMO_TOKEN=local-demo-token ./bin/aegis-api

In another terminal: python3 scripts/acceptance.py Or build then run the lifecycle-managed experiment script: python3 scripts/local_experiments.py Script requires port 8080 free; overwrites local raw/ experiment files. Normal result includes task_id, SUCCEEDED, steps, and evidence_ids; failure scenarios show FAILED with explicit error_code; cancellation yields CANCELLED. Do not treat HTTP 202 on create as execution success.

For API-only observation, use README curl commands to create a task, then query with the returned task_id. Cancelling an already-completed task returns the original terminal state — history is not rewritten. Sending an Idempotency-Key explicitly returns an unsupported error: W01 has no idempotency TTL or parameter conflict comparison, so it cannot silently accept a header that would create false expectations.

9. Week One's Engineering Foundation and Open Problems

This week's value: placing the agent's dynamic decisions inside deterministic boundaries — every task has queryable state; every model response has a defined structure; every tool call passes independent validation; every loop has a finite budget; every exception has a stable termination reason; every cancellation is protected by terminal-state guarantees.

Suitable for local teaching, adapter contract development, deterministic fault testing. Not suitable for environments requiring process-crash recovery, cross-instance task scheduling, real cluster changes, or enterprise identity governance. No PostgreSQL, checkpoint, outbox, leases, MCP, approval, OTel, or real LLM yet. A process crash loses all in-memory state and queue; the restart experiment confirmed this boundary.

W02 will add MCP read-only queries behind ToolExecutor, validating schema, protocol, and cross-process failure paths. W04 will swap the task repository for a persistent implementation with transactions, checkpoints, outbox, and claim mechanisms. Replaceable interfaces do not automatically confer reliability; those semantics need new implementations and new fault experiments.

For readers moving from Java backend to agent engineering, the same judgment rules apply: business state needs controlled updates; long-running ops need independent lifecycles; authority cannot come from the requester's self-assertion; retries must understand side effects; critical conclusions need reproducible evidence. The model's non-determinism makes these backend fundamentals even more worth stating explicitly.