Why Build AI Agents? Benefits, Challenges, and Real-World Examples

1. What Is an Agent?

An Agent is a system that lets a large language model (LLM) act as a proxy for human behavior, using tools and APIs to accomplish tasks. OpenAI defines an Agent as LLM + Planning + Memory + Tool Use. The Fudan NLP team describes it with three components: Brain, Perception, Action , where the brain handles memory, reasoning, and decision‑making, perception processes multimodal input, and action executes tool calls.

2. Why Build Agents? Advantages

Lower development threshold : Non‑developers can create functional applications by describing prompts, eliminating the need for hand‑coded solutions.

Simplified workflow complexity : The LLM acts as “glue” that automatically maps outputs of one API to inputs of the next, reducing the need for exhaustive parameter conversion and validation.

Rich interaction modalities : Agents are not limited to pure text; they can handle GUI, multimodal inputs, and generate structured outputs such as charts or tables.

Collaborative complex‑task execution : Multiple agents can be assembled, cooperate, or even compete to solve multi‑step problems, enabling expert‑like decision making.

Examples include ByteDance’s Jianying video editor, which uses AI templates to let anyone edit videos, and Meitu’s photo app that offers one‑click beautification via AI tools.

3. Challenges of Agents

Slow response time : Agents rely on streaming LLM outputs, which can cause multi‑second latency, especially with long prompts or complex reasoning steps.

Hallucinations : LLMs may produce factual errors or ignore instructions, leading to trust issues.

Unfriendly pure‑text interaction : Long, verbose textual responses can be hard for users to read compared with structured UI elements.

Mitigation strategies include hardware acceleration (GPU, AI chips), software optimizations such as FlashAttention, vLLM KV‑Cache tricks, model pruning, distillation, quantization, and prompt engineering (meta‑prompting, system‑2 reasoning, GraphRAG).

4. Multi‑Agent Collaboration

Modern research explores Multi‑Agent systems where agents can be assembled, cooperate, or compete. Scenarios include:

Sequential handling of multiple user queries in a service ticket by invoking specialized agents.

Expert‑panel style decision making where several domain‑specific agents propose solutions and a coordinator selects the best.

Future visions of an “Agent society” where agents perform distinct roles (e.g., cooking, music performance) and humans can interact at any stage.

5. Conclusion

Although current agents face speed and hallucination issues, continuous advances in hardware, model optimization, and prompting are steadily reducing these drawbacks. The benefits—lowered development cost, simplified workflows, versatile interaction, and collaborative capabilities—make building agents a net positive investment for the future.

References

Lilian Weng. LLM Powered Autonomous Agents.

Xi, Zhiheng, et al. The Rise and Potential of Large Language Model Based Agents: A Survey.

Anthropic. Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku. https://www.anthropic.com/news/3-5-models-and-computer-use

Microsoft Blogs. New autonomous agents scale your team like never before. https://blogs.microsoft.com/blog/2024/10/21/new-autonomous-agents-scale-your-team-like-never-before/

Suzgun, Mirac, and A. T. Kalai. Meta‑Prompting: Enhancing Language Models with Task‑Agnostic Scaffolding.