Why Powerful AI Agents Are Becoming More Like Traditional Software

Palantir presented at DevCon 6 a production‑grade Agent Stack. The Orchestrator provides durable execution, the Agent Engine/SDK supplies the runtime, the Agent Manager handles telemetry and observability, and AIP Evolve optimizes models, prompts and the whole workflow.

The architecture reflects a shift: short‑lived agents that only need model + prompt + tool + loop are sufficient for seconds‑scale tasks, but when agents run for hours, days or weeks the system must manage state, recovery, waiting, permissions, audit and external failures. The difficulty moves from “what to infer next” to “can the agent work reliably over time”.

The Agent Loop is modeled as a distributed state machine. The Engine breaks execution into three primitives—Context Item (typed session state), Event (state‑transition trigger) and Effect (asynchronous call to model, API or other services). An Event enters the system, a handler updates state and may fire Effects; Effects produce new Events, forming a chain of state migrations while the model remains responsible only for inference.

Durable execution is essential for long‑running tasks. The Orchestrator must survive crashes, pauses and external API time‑outs, allowing agents to resume from the last known state. This brings classic distributed‑system concerns—retry, idempotency, event ordering, permission checks and observability—into the AI stack.

Observability is addressed by the Object Timeline feature, which attaches Agent actions, token usage, runtime, waiting time and “Agentic Coverage” to the business object’s timeline, enabling attribution of each modification to an Agent, a human or an external system.

AIP Evolve adds a layer of automated evaluation and optimization. Specialist “FDE” agents analyze, test, replace models and refine prompts. In Palantir’s example, swapping GPT‑4o for GPT‑5.4 Mini and adding guardrails kept functional correctness on ten test cases while cutting average compute cost from 204.6 to 72.4 seconds per call (≈ 65 % reduction). AIP Evolve is currently in beta.

Ontology connects the internal Agent state to real‑world business entities, defining objects, relationships, actions, permissions and audit trails. This turns the Agent from a memory‑only component into a system that operates on governed business state, making the execution traceable and auditable.

The overall conclusion is that modern AI agents are no longer just smarter models; they are becoming distributed software systems that must provide state management, durable execution, observability and governance. Palantir’s stack embodies this shift by extending the classic software stack with State, Event, Effect, Orchestration, Observability and Ontology.