Tech Briefing #2 2026: AI Agent Evals, vLLM Memory Leaks & Software Complexity

1. Demystifying evals for AI agents

Anthropic's comprehensive guide to AI agent evaluation, providing complete definitions, methodology, and practical guidance for assessing agent capabilities.

Source: https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

2. The Second Half

Emphasizes the importance of real-world task evaluation over narrow benchmarks. The shift is from "Can we train a model to solve X?" to "What should we train AI to do, and how do we measure real progress?"

Two key approaches:

Develop novel evaluation setups or tasks for real-world utility.

Solve them with the recipe or augment the recipe with novel components.

Example: Language generalizes through reasoning in agents. Task-specific RL has limited prospects, but generalizable RL via LLM reasoning becomes feasible, such as decomposing tasks into discrete operations.

Source: https://ysymyth.github.io/The-Second-Half/

3. 21 Lessons From 14 Years at Google

Addy Osmani's career lessons for early-career engineers, aimed at smoothing professional growth and achieving higher long-term impact.

Source: https://addyosmani.com/blog/21-lessons/

4. Why is calling my asm function from Rust slower than calling it from C?

Deep performance optimization analysis tracing the issue down to compiler behavior, assembly-level details, and data cache misses.

Source: https://ohadravid.github.io/posts/2025-12-rav1d-faster-asm/

5. Skills library

Anthropic's Skills repository for agent capability development.

Source: https://github.com/anthropics/skills

6. How MCP and tool calls work

Explains how LLM inference engines — which only predict and sample output tokens — actually invoke tools. The article provides a clear interpretation of the mechanism, covering the process observed in frameworks like CrewAI.

Source: https://virtuslab.com/blog/ai/how-mcp-llm-tool-calls-work/

7. Logging sucks

Analyzes shortcomings of conventional logging (optimized for writing, not querying) and proposes structured logging design methods, particularly valuable for complex systems.

The fundamental problem: logs are optimized for writing, not for querying.

Source: https://loggingsucks.com/

8. The bitter lesson: The less you build, the more it works

Argues against complex multi-agent engineering platforms; advocates letting the model handle more, keeping agents as simple as possible. References the browser-use/agent-sdk project.

Source: https://github.com/browser-use/agent-sdk

9. Codex agent technology

Detailed walkthrough of the codex.cli agent loop, serving as excellent material for understanding how an agent operates. Performance considerations include:

previous_response_id reuse

prompt caching

context compression techniques

Source: https://openai.com/index/unrolling-the-codex-agent-loop/

10. Why We've Tried to Replace Developers Every Decade

Historical review from Apollo guidance software through COBOL, CASE tools, Visual Basic, to modern AI. Each wave improved efficiency but failed to solve essential complexity — integration with existing systems, security, performance under load, long-term maintenance. Complexity emerges in the details.

complexity—integration with existing systems, security considerations, performance under load, long-term maintenan。complexity emerges in the details.

The artifacts we create — whether COBOL programs, Delphi forms, or Python scripts — are the visible outcome of invisible reasoning about complexity . We just keep discovering that the constraint isn't the tool — it's the complexity of the problems we're trying to solve.

Source: https://www.caimito.net/en/blog/2025/12/07/the-recurring-dream-of-replacing-developers.html

11. Observability's Past, Present, and Future

Observability arises from software complexity, especially distributed systems where no single engineer holds a complete mental model. Probabilistic issues (performance, reliability) demand observable systems. In the AI era, code volume may surge and technical debt grow ("X mountain of code"), making observability even more critical for maintainability and control.

Source: https://blog.sherwoodcallaway.com/observability-s-past-present-and-future/

12. Debugging a memory leak in vLLM

Step-by-step investigation of a vLLM memory leak on Linux:

Ruled out Python-level leaks.

Used glibc's heaptrack to exclude standard heap leaks.

Examined RSS/PSS metrics to suspect mmap-type leaks.

Traced mmap allocations with bpftrace, discovered missing frame pointers.

Set gdb breakpoints to pinpoint the culprit to a communication library.

Source: https://mistral.ai/news/debugging-memory-leak-in-vllm

13. Hotspot performance engineering fails

Questions why teams accumulate hundreds of micro-optimizations instead of identifying the single function consuming 80% of runtime. Aligns with recent Android ART VM optimization findings and Jeff Dean's performance tips.

Also referenced in technical briefing 2025 issue 9 (year-end): https://zhuanlan.zhihu.com/p/1985067755452518594

Source: https://lemire.me/blog/2023/04/27/hotspot-performance-engineering-fails/

14. Why LLMs are non-deterministic

Beyond temperature-based token sampling, non-determinism persists even at temperature 0 due to floating-point computation variances, GPU concurrency issues, and other low-level factors.

Source: https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/