Improving Observability in Multi‑Agent Systems: Analysis and Extension of OpenAI Swarm

The presentation begins with an overview of multi‑agent systems (MAS), highlighting their ability to solve problems that single agents cannot handle and noting that current open‑source MAS projects—Swarm, Auto‑GPT, MetaGPT, LangChain, and LangGraph—integrate multi‑agent capabilities to varying degrees.

It then discusses the observability challenges inherent in MAS, emphasizing that complex internal routing, black‑box LLM components, and dynamic workflow execution make it difficult to infer system state from external outputs, which hampers debugging, testing, and reliability.

OpenAI Swarm is introduced as a lightweight, Python‑based framework with roughly 500 lines of core code, compatible with OpenAI APIs and other models. Its simplicity offers a clear view of agent scheduling but lacks built‑in memory and comprehensive observability features.

The authors describe their work to improve Swarm’s observability: enabling debug logging to capture intermediate messages, aggregating agent responses into a unified history, persisting function‑call data via a global context variable, and storing all logs in JSON files for later analysis.

Based on the collected data, they built the “AgentInsight” service, which visualizes agent interactions as chat‑style dialogs, flow‑charts, and war‑room dashboards, allowing users to see which agents were invoked, execution times, and tool usage.

Future plans include extending support to other MAS frameworks (MetaGPT, AutoGen, etc.), adding richer visualizations (dynamic graphs, replay‑and‑continue functionality), and providing front‑end drag‑and‑drop workflow editors to define and modify agent pipelines.

The session concludes with a Q&A covering the distinction between workflows and MAS, the role of agent frameworks versus model capabilities, and the contributions algorithm engineers can make to multi‑agent collaborations.