What I Learned Moving from Backend Engineering to AI Agent Development

LLM Uncertainty Is an Inherent Feature

LLM output variability stems from floating‑point precision errors (FP16/BF16), heterogeneous hardware (different GPU models), Mixture‑of‑Experts routing, sampling strategies, and other factors. Even with Temperature=0 the output is not fully stable because model providers trade exactness for lower cost and higher throughput during inference; this uncertainty cannot be eliminated, only understood and accepted, underscoring the importance of Agent engineering.

Agent Development Is Fundamentally Context Engineering

The core of Agent development is how to inject business knowledge into the LLM’s context. Knowledge internalization occurs in three stages: pre‑training (general knowledge), fine‑tuning (domain knowledge), and Prompt/RAG (real‑time knowledge). Most AI Agents only touch the third stage. The quality of the Context layer directly determines an Agent’s productivity and robustness, making it critical.

Trend of Application‑Layer Code

In traditional micro‑service architectures, extensive business logic resides in code (state machines, exception handling, transactions). After introducing LLMs, this complex logic is gradually absorbed into model weights, turning application‑layer code into “glue” that connects prompts, tool calls, and output parsing. This trend will intensify as foundational models become more capable.

Workflow Value May Be Underestimated

Using LangChain as an example, its real value lies not in DAG orchestration (ReAct is essentially while + if, LangGraph is conditional branching) but in defining a semantic, standardized interface for AI Agent development—clearly separating SystemMessage, UserMessage, ToolMessage, etc. This creates a consensus protocol between developers and model providers, similar to the early Java SSH framework, though its dominance may shift as large‑model capabilities evolve and each vendor offers its own Agent framework.

Changing Software Failure Modes

Traditional distributed systems fail due to deterministic resource congestion (CPU spikes → timeout → cascade), mitigated by circuit breaking, throttling, and scaling. Agent systems fail through error‑probability propagation (single‑step inference bias → erroneous premise → subsequent steps all wrong), which is hard for conventional monitoring to capture. Real‑time quality‑assessment feedback loops are required, representing another key aspect of Agent engineering.

ReAct Paradigm and Closed‑Loop Feedback

The ReAct paradigm resembles classic closed‑loop control: the LLM acts as the controller, tools as actuators, and observations as sensor feedback. Currently, observations are mostly derived from LLM outputs (probability × probability) and have not truly left the semantic space. The author envisions more reliable solutions that anchor feedback to physical world impacts (e.g., real traffic changes, device states), though substantial work remains.

Problems Suited and Unsuitable for LLMs

Suitable scenarios include:

Unstructured input → structured output (contract review, meeting minutes) where rule‑based methods cannot exhaustively cover the input space.

Fuzzy retrieval and semantic alignment (self‑service Q&A, cross‑document semantic comparison).

Long‑tail problems requiring probabilistic fallback (high‑dimensional inputs, non‑convergent contexts).

Unsuitable scenarios are deterministic workflow orchestration, low‑latency high‑throughput transactions, and tasks demanding strong transactional consistency.

Conclusion

In the AI era, business logic is shifting from the application layer into model weights, moving from deterministic code to probabilistic parameters, and from manually designed structured complexity to emergent black‑box complexity. The core competency of AI Agent engineers is therefore “understanding and mastering uncertainty.”