From Prompt Engineering to Graph Engineering: The Evolution of AI App Development Paradigms

Writing good prompts used to be the most intuitive part of developing large-model applications. Now developers must also consider what information the model receives, which tools it can use, how to handle mid-execution errors, and how the overall task concludes.

Introduction

Early interactions with large models were simple chat: ask a question, get an answer. Real applications require the system to retrieve data, call tools, verify results, and handle exceptions. Tasks may be interrupted mid-way and need to resume from the same point. The developer's focus has shifted from "how to ask the model" to "how to make the whole system finish the job."

Prompt Engineering, Context Engineering, Harness Engineering, Loop Engineering, and Graph Engineering each address engineering problems that emerge as an application evolves from answering questions to executing tasks. They are not successive replacements, nor must every project use all five.

Prompt Engineering

When an application only needs a single input and output, the first concern is how to describe the task so the model does not drift. Classification, extraction, Q&A, and content generation all require clear requirements.

Prompt Engineering revolves around system instructions and user prompts: define task boundaries, provide examples, enforce output formats, and make model responses stable and usable.

Use clear instructions to specify goals, constraints, and evaluation criteria.

Show few-shot examples mapping inputs to ideal outputs.

Require structured output such as JSON.

Break complex tasks into multiple smaller model calls with programmatic validation in between.

These techniques reduce misunderstanding, but even perfect instructions cannot help if the model lacks relevant data. For instance, asking why an order is abnormal without providing order records and system logs yields limited improvement. Developers then ask: what information should be given to the model on each call?

Context Engineering

Model calls are stateless; the context for a call includes not only the prompt but also prior user messages, retrieved documents, tool results, and the current task step. Context Engineering manages how this information enters the model's context window.

Context windows are limited, and more data is not always better. Dumping entire documents can drown key details like order IDs or error logs. A practical approach is to select information based on the current step: provide order records and logs during troubleshooting, then supply resolution conclusions and communication guidelines when replying to the user.

Use RAG to retrieve relevant documents for the current question instead of stuffing the whole knowledge base into the prompt.

Separate short-term conversation, long-term memory, and business data.

Summarize or compress overlong history.

Load tool descriptions and skill documents on demand.

Reassemble context before each execution round based on current state.

Harness Engineering

Harness is the "scaffold" that connects the model, context, tools, and execution environment, and manages the invocation process. Designing around this scaffold is Harness Engineering.

Example: a code agent fixing a bug must first read the repository, search code, then edit files and run tests. Read/write permissions and commands requiring approval must be defined upfront. The harness wires these capabilities and rules together: the model initiates tool calls, the runtime executes and returns results, and an audit trail is kept.

Unify model invocation and tool interfaces.

Validate tool parameters and results with schemas.

Manage identity, permissions, sandboxes, and human approval.

Log, trace call chains, set timeouts and budgets.

Persist session and task state, handle retries and exceptions.

Loop Engineering

The "act–observe–act" cycle is what Loop Engineering handles. The model chooses an action, the runtime calls a tool, returns the real result, and the model decides whether to continue, adjust, or stop. Loops need limits on rounds, time, and budget.

Example: a user asks why an order is delayed. The agent checks order status, then logistics records; finding logistics stalled, it queries warehouse outbound records. It narrows down step by step and answers based on evidence, not guessing the whole process in the first call.

Alternate model calls and tool calls in a loop.

Append each tool result to the conversation or task state.

Set maximum rounds, timeouts, budgets, and explicit stop conditions.

Retry tool failures; let the model adjust its plan when needed.

Use an evaluator–optimizer pattern where check results trigger the next revision.

Graph Engineering

When a task involves querying multiple systems in parallel, undergoing tests and human approvals, and recovering from interruptions, embedding all that logic into an agent's prompt or internal loop makes it hard to see which steps are mandatory and where execution returns after failure. The task itself needs a clear execution graph.

Graph Engineering organizes steps, states, and transitions into an execution graph. Nodes can be plain code, a model call, an agent with tools, or a human approval step; edges define allowed transitions. Previously retrieved information and execution results are stored in task state for downstream nodes.

The graph here refers to an execution graph describing how a task runs, distinct from knowledge graphs or GraphRAG graphs that describe entities and relationships.

Explicit graphs are useful when tasks have multiple possible paths, parallel processing, human review, or interrupt–resume requirements. For single-turn Q&A or short tasks completed by one agent, this layer is usually unnecessary.

Graphs do not require multiple agents; any need to clarify step dependencies, checkpoints, and state transitions can use pure code nodes alongside agent nodes.

Take "order stuck in pending, find cause and fix" as an example. The system first collects order ID and timestamp, then queries logs, traces, and recent deployments in parallel, merging results into a single task state.

A diagnosis node judges whether evidence is sufficient. If not, it triggers supplementary queries; if a code defect is found, it branches to patch generation; if a recent deployment is the culprit, it prepares a rollback. Both patches and rollbacks pass through permission and approval checks. After execution, tests run and metrics are observed; verification failure returns to diagnosis, success ends the task.

Task state retains queried logs, current hypotheses, fix records, approval and verification outcomes. With checkpoints configured, an interrupted task can resume from the saved position without re-collecting evidence. Frameworks like LangGraph provide primitives for state, loops, and checkpoints.

One might ask how this differs from workflows. There is no hard boundary; workflows already support branching, rollback, parallelism, and human approval. Graph Engineering simply makes the task's steps, states, and transitions explicit in a graph to visualize the full execution path.

The graph can enforce rules: no repair without approval, no completion without verification, escalate to human after too many retries. But drawing nodes and edges is not enough; actual enforcement requires programmatic checks, permissions, and runtime guards.

Within these guardrails, the agent can read logs, decide what to investigate next, choose tools, and switch strategies after a failed fix. It may even iterate repeatedly inside a single node. Transitions like "verification failed → return to diagnosis" are executed by code; whether the failure is due to a bad patch or an unstable test environment can be judged by the agent based on evidence, with its suggestion then validated by code.

This design does not mandate a dedicated graph framework; workflow engines, state machines, or plain code can implement it. Graph Engineering's value is expressing paths and dependencies clearly when they become complex.

In such fault-handling tasks, I prefer a fixed outer control plane and an open inner intelligence plane: approvals before repair, mandatory verification after, and programmatic handling of failure escalation; while investigation direction, tool usage, and fix strategies are left to the agent to adapt to the live situation.

How the Five Engineering Concerns Connect

These methods often coexist in one application. In the fault-diagnosis graph: the diagnosis node writes prompts (Prompt Engineering), pulls relevant logs as context (Context Engineering); Harness connects tools and permissions; the agent loops internally (Loop Engineering); Graph arranges subsequent testing, approval, and verification.

A simple text classifier may need only a clear prompt and output validation. Add external knowledge → handle Context; need tool calls → build Harness; task repeats → design Loop; when paths, state, and recovery grow complex → organize into an explicit graph. How far you go depends on the task itself.

Conclusion

From writing prompts to designing graphs, the problems become more concrete: does the model understand the task? Does it have the needed information? Can it call tools? How does it continue after an action fails? How do multiple stages hand off? Each step forward requires developers to add system design that complements the model's capabilities.

Whether an application works well is judged not by how many engineering buzzwords it uses, but by whether the task completes and whether problems can be diagnosed and resumed. That is where this evolution truly lands in engineering practice.