15 Open-Source Tools for Production AI Agents: Frameworks, Workflows & Execution

The article surveys 15 open-source projects that are used in production AI agent systems, grouping them by function: building agents, running persistent workflows, connecting and executing tools, observing and improving systems, adding research capabilities, and referencing complete applications. The author emphasizes that tool choice depends on the specific task and stability requirements, and that public adoption cases or dependency listings do not constitute safety guarantees — deployers must still verify permissions, isolation, resource limits, and operational responsibilities.

Main Tools and Their Roles

Building agents: LangGraph and Pydantic AI provide stateful control flow, typed tools, and application integration.

Persistent workflows: Temporal handles recoverable background workflows, documented in Sanity's Content Agent architecture.

Connecting tools and executing tasks: Pydantic (data contracts), MCP Python SDK (tool interfaces), Playwright (browser control), Browser Use (task-driven browser automation), and E2B (isolated execution environments).

Running and improving systems: Langfuse (observability), promptfoo (testing and red-teaming), and LiteLLM (model gateway) cover distinct responsibilities and are not interchangeable.

Research capabilities and reference applications: Valyu TypeScript SDK offers research APIs; Global Hack Map, Global Threat Map, and Polyseer demonstrate domain-specific interfaces and outputs.

1. LangGraph: Explicit State and Control Flow Management

LangGraph orchestrates stateful agents with graph-based control flow, checkpoints, and human-in-the-loop pauses. Public deployments include LinkedIn's recruiting assistant, Uber's code-testing workflow, and AppFolio's property-management Copilot. These workflows span multiple stages and tools, requiring execution state persisted independently of the latest model response.

If an agent needs branching, human approval, or resuming interrupted tasks, LangGraph is not a turnkey solution; developers must first decide which state must be durably saved and then encode state transitions in application code. Checkpoints should be written to persistent storage, and external operations must be safely retryable. Restoring graph state alone does not prevent duplicate payments or duplicate message sends.

2. Pydantic AI: Typed Agents for Python

Pydantic AI supports typed tools, structured output, and dependency injection for Python agents. The Pydantic validation library and Pydantic AI are separate projects. MindsDB's migration case study notes adoption of Pydantic AI to control agent state and validate intermediate results programmatically. The marimo project's dependency list includes Pydantic AI Slim for its AI features.

Typed output is especially convenient when components pass results to each other: application code can inspect fields directly and decide next actions without asking the model to re-read and interpret text. However, structural validation is not enough — business correctness and permissions must be checked separately. A supplier ID may match the field type yet point to the wrong object; an action parameter may conform to the schema but the caller may lack permission to execute it.

3. Temporal: Recovering Long-Running Workflows

Temporal separates workflow coordination from the Activities that perform external work, using recorded execution history for fault recovery. Sanity's deployment case describes Temporal's role in content-schema graph computation and parallel background modification tasks that support its agent workloads without keeping chat requests online.

Cross-service calls, waiting for events or approvals, and resuming after worker failures are typical Temporal use cases. Users can submit a task, leave the application, and check progress or results later. Before going live, teams must understand workflow code determinism requirements, Activity retry and versioning policies. Side effects on external systems still need idempotency guarantees. Self-hosting Temporal versus using Temporal Cloud carries different operational responsibilities.

4. Pydantic: Shared Dependency Across Model Providers

Pydantic provides typed data models, validation, serialization, and JSON Schema generation. Dependency manifests show that the official Python SDKs of OpenAI, Anthropic, and Google, as well as the OpenAI Agents SDK, all declare Pydantic as a dependency.

Tool requests, model responses, and API results can be constrained by explicit data contracts before entering business logic, enabling testing of valid and invalid fields and letting downstream components process results with known structure. Again, data structure, business correctness, and operation permissions must be verified independently. These SDK dependencies only reflect public software usage and cannot be used to infer the internal inference architectures of the model providers.

5. MCP Python SDK: Reusable Tool Interfaces

The official Model Context Protocol Python SDK enables building MCP clients and servers. The OpenAI Agents SDK lists the mcp package as a runtime dependency, and Browser Use declares the same. These downstream dependency declarations are more concrete evidence of actual usage than product compatibility lists.

A unified tool interface lets a single service serve multiple clients, reducing the effort of building per-framework adapters. Implementers must still add authentication, authorization, input validation, and tenant isolation. MCP standardizes the communication interface, but access to customer data remains the application's responsibility. Compatible SDK versions must be chosen and security updates monitored continuously.

6. Playwright: Programmatic Browser Control

Playwright provides browser automation for navigation, interaction, downloads, and inspection. OpenClaw's dependency manifest lists playwright-core as a runtime dependency. Microsoft's Playwright MCP implementation also depends on Playwright. Both are verifiable downstream dependencies, but they do not imply that all agent tasks invoke a browser.

When website flows are relatively fixed, programmatic page transitions, file downloads, and result checks can be scripted and handed to subsequent components. Such flows are typically easier to test than letting the model decide click positions each time. Browser sessions require isolation; login credentials and session state must be protected. High-impact operations should include approval steps. Some browser agents use custom Chrome DevTools Protocol drivers, so not all can be assumed to run on Playwright.

7. Browser Use: Task-Goal-Driven Browser Operation

Browser Use supplies a browser-agent development library with supporting tools and operates a separate cloud service. Hermes Agent's documentation explicitly records this dependency: Hermes's browser execution tool uses Browser Use's browser-harness driver and lists it as a regular Python dependency. The docs also explain how model-generated code controls a local browser or a pre-configured cloud browser.

When site structure is unfamiliar, the UI changes frequently, or task steps cannot be fully scripted in advance, Browser Use can drive the browser. The calling application must still constrain access and operation permissions. Web content should be treated as untrusted input, with target sites and execution scope limited. Read-only retrieval versus purchases or account modifications should have different permission levels. Choosing a hosted browser versus the local library affects deployment and management.

8. E2B: Execution Environments for Agents

E2B offers an SDK for creating and managing isolated execution environments, suitable for building code interpreters. Genspark's public case study describes how Genspark uses E2B for code execution, file handling, and multi-step tasks. E2B also publishes customer cases involving Gumloop and Lindy.

Model-generated code needs a place to run; isolated environments prevent it from gaining direct web-server permissions. A data-analysis agent can process uploaded files in a sandbox and return analysis artifacts to the user. Configuration must specify which credentials, data, and network resources the sandbox can access, and enforce resource quotas and runtime limits. Published cases mainly involve E2B's hosted service and do not prove those customers also self-host the open-source components.

9. Langfuse: Tracing and Evaluating Agent Behavior

Langfuse provides distributed tracing, prompt management, evaluation, and datasets for model applications. Canva's public case cites two scenarios: a Java-based help assistant and a Python async customer-service agent. The team uses Langfuse to record call chains, manage prompt versions, and run online and offline evaluations, later migrating from self-hosted to Langfuse Cloud.

When an agent answers incorrectly, traces help developers step through retrieval, model calls, and tool execution to locate the failing segment. Typical failure cases can be saved as test sets for regression checks before new releases. Before logging user conversations and tool results, teams must define PII redaction rules, access controls, and log retention periods. Tracing should cover application logic; recording only final outputs often misses the intermediate step where the error occurred.

10. promptfoo: Evaluation and Adversarial Testing

promptfoo supplies a CLI and library for evaluating and red-teaming model applications, with support for automated test pipelines. Its public customer statements list OpenAI and Anthropic; this information comes from promptfoo's own announcements without corresponding public dependency manifests or internal deployment details. Evaluation tools used for production products may not directly handle live user requests.

Teams can turn observed failures into repeatable test cases covering normal requests, hallucinated answers, and operations that should be rejected. After each prompt, model, or tool implementation change, the suite is re-run. Evaluation must check what the system actually did; if the integration layer provides tool-call logs, they should be included. A plausible-looking answer does not guarantee correct operation. Passing a test suite only means the system met criteria for those cases, not that it resists all attacks. Teams must also verify where evaluation data is sent.

11. LiteLLM: Unified Model Access and Routing

LiteLLM provides a unified model interface and proxy service with request routing, virtual keys, cost tracking, and load balancing. NVIDIA's public testimonial describes the need for a consistent interface to access cloud models, open-source deployments, and internal model services, indicating LiteLLM can serve as a shared model gateway, though it does not prove all NVIDIA agents use LiteLLM.

Managing multiple applications, a unified gateway simplifies per-project usage accounting, centralizes provider configuration, enables consistent routing policies, and reduces per-client troubleshooting. Fallback models used during failover must be validated with actual tasks. API consistency does not guarantee identical tool-calling ability or response quality; feature-specific licensing must be checked. Cost dashboards provide visibility, but whether budget enforcement actually blocks overspending requires separate testing.

12. Valyu TypeScript SDK: Adding Research to Existing Agents

The Valyu TypeScript SDK, published as valyu-js, gives JavaScript and TypeScript applications typed search, content extraction, and asynchronous DeepResearch interfaces. Public code shows a concrete example: Global Threat Map's server-side client imports the SDK to fetch events and run research tasks. This confirms the SDK's usage in one Valyu application but does not indicate market share.

When preparing a company research brief, an application can call the SDK for retrieval or launch a long-running research task and read results on completion. Task IDs, execution status, and source links should be stored alongside results so users can review the research process and verify citations. API keys must stay server-side, call volume should be limited, and incomplete or failed research tasks handled gracefully. The SDK being open-source does not mean Valyu's hosted API is free, nor that the hosted service can be self-deployed.

13. Global Hack Map: Security Monitoring Interface

Global Hack Map displays vulnerabilities, exploits, and data breaches on a map. The dashboard separates newly emerged critical vulnerabilities from reported exploitation activity and tags events as confirmed, reported, or merely claimed.

Security teams responsible for agent infrastructure can first check which developments affect their tech stack, then follow links to advisories or reports to decide whether to escalate investigation. Global Hack Map places this triage workflow on a map and dashboard for risk-based filtering. When verifying events, teams should return to original security advisories to confirm affected software versions and exploitation status. News reports do not prove one's own systems are compromised, and map coordinates alone cannot prove attack origin.

14. Global Threat Map: Geographic Context for Research

Global Threat Map combines a map, event feed, country research, and entity profiles in a single situational-awareness application. Its repository describes sourced research functionality and report export capability.

For example, a regional risk analyst spots an event in the feed, reviews the original source, generates a research profile for the relevant country or organization, and compiles a briefing. The entire process — retrieval, location data, and long-running research tasks — resides in one interface, with reports exportable for colleague review. The repository serves as an implementation reference ; it does not provide evidence that other production agents depend on this project.

Before deployment, verify geocoding accuracy, source update frequency, and whether existing API integrations still apply. The documented self-hosted mode lacks authentication; API calls consume the deployer's quota, so access control and rate limits are needed before sharing. A Mapbox token must also be configured.

15. Polyseer: Structured Research for Prediction Markets

Polyseer accepts Polymarket or Kalshi market links, reads the market question, price, deadline, and settlement rules, then submits a research task to Valyu DeepResearch. Its server-side implementation handles market data fetching, research task creation, and progress tracking. According to project documentation, output includes structured predictions, supporting and opposing evidence, and downloadable reports. Analysis must also cross-check assumptions against the market's actual settlement rules.

Polyseer's implementation uses market data as the research task boundary, with asynchronous evidence collection and a fixed task ID for result retrieval. Validation should use already-settled markets to check prediction quality. A model outputting a probability value does not mean the probability forecast is calibrated. The documented self-hosted mode has no authentication; research requests use the deployer's configured Valyu API key, so access and call costs must be limited before opening to others.

Summary

Which open-source agent framework to start with? Applications favoring typed Python code can try Pydantic AI first; those needing explicit control over state, branching, and recovery flows can try LangGraph first. Before committing, run a real task end-to-end and test potential failure points. Research SDKs and reference applications solve different problems and do not necessarily require switching agent frameworks.

Does production use mean safe to deploy directly? No. Public adoption cases or dependency listings only indicate reported usage or declared dependencies, not safety guarantees. Deployers must still verify permissions, isolation, resource limits, and actual runtime behavior; managed services and self-hosted options have different responsibility boundaries.