How OpenAI Turns Models into Agents by Adding a Computer Environment to the Responses API

Shell Tools

A compact execution loop starts with the model proposing an action (e.g., read a file or call an API), the platform runs the action, and the result feeds back into the next step. The simplest loop uses shell tools, which let the model interact with a Unix‑like command line. The provided shell toolset includes grep, curl, and awk, enabling tasks that go beyond the Python‑only code interpreter, such as running Go or Java programs or starting a NodeJS server.

Orchestrating the Agent Loop

The model can only suggest shell commands; an orchestrator must capture the model’s output, invoke the tools, and feed the tool responses back to the model until the task finishes. The Responses API hands control back to the client when custom tools are used, but it can also natively coordinate model‑tool interactions.

When the API receives a prompt, it assembles the model context (user prompt, prior conversation state, and tool instructions). The model must be trained to emit shell commands—GPT‑5.2 and later meet this requirement. The API forwards each command to a container runtime, streams the shell output back to the model, and repeats the cycle until the model returns a final answer without additional commands.

The API streams output in near‑real time, allowing the model to decide whether to wait for more data, issue another command, or produce the final response. Multiple commands can be issued in a single step; the API runs them in parallel container sessions, multiplexing the streams into structured tool output.

Because raw shell output can be large, the model specifies an output limit for each command. The API enforces the limit, returning a bounded result that keeps the beginning and end of the output while marking the omitted middle, e.g.,

text at the beginning ... 1000 chars truncated ... text at the end

. This bounded, parallel execution keeps the agent loop fast and context‑efficient.

Context‑Window Compression

Long‑running agents can fill the model’s context window, jeopardizing cross‑turn reasoning. To preserve important details while discarding noise, OpenAI added native compression to the Responses API. The latest models can analyze prior dialogue, produce a compact token representation, and prepend it to the next window together with high‑value recent tokens.

Compression can be invoked server‑side automatically or via the /compact endpoint, where developers can set thresholds. The server handles timing, allowing slightly oversized inputs to be processed and compressed rather than rejected. Compression capabilities evolve with each new model version.

Container Context

Containers serve as both execution hosts and the model’s working context. Inside a container the model can read files, query a database, and access external systems under controlled network policies.

File System

A container‑based file API lets the model map available data, choose target files, and avoid noisy scans. Instead of packing all inputs into the prompt, resources are staged in the container file system for on‑demand access via shell commands.

Database

Structured data should be stored in a database such as SQLite. The model receives a table schema description and can query only the rows it needs, e.g., “Which products had sales decline this quarter?”—avoiding the cost of scanning an entire spreadsheet.

Network Access

Outbound network requests flow through a centralized proxy that enforces allow‑lists and access controls. Credentials are injected at the domain level, remaining invisible to the model while still enabling authenticated calls. This design reduces the risk of data leakage while preserving useful internet access.

Agent Skills

Repeated multi‑step patterns are packaged as reusable "skills"—folders containing a SKILL.md metadata file and any supporting resources (API specs, UI assets). The runtime can discover a skill via shell commands ( ls, cat), interpret its instructions, and execute its script within the same agent loop.

Skills are uploaded as versioned packages and retrieved by skill ID. Before sending a prompt, the Responses API loads the skill and injects its metadata and path into the model context. The deterministic loading sequence is:

Fetch skill metadata (name, description).

Copy the skill package into the container and unpack it.

Update the model context with the metadata and container path.

The model then explores the skill’s instructions step‑by‑step, executing shell commands as needed.

Putting It All Together

Responses API provides orchestration, shell tools supply executable actions, hosted containers give persistent runtime context, skills add reusable workflow logic, and compression keeps long‑running agents within the context window. Together, a single prompt can drive an end‑to‑end workflow: discover the right skill, fetch data, transform it into structured state, query efficiently, and produce persistent artifacts such as spreadsheets.

The accompanying diagrams illustrate how real‑time data is turned into a spreadsheet through this system.

Build Your Own Agent

Developers can follow OpenAI’s blog posts and cookbooks for step‑by‑step guidance on packaging skills and invoking them via the Responses API. The platform continues to evolve, aiming to handle increasingly complex, large‑scale real‑world tasks.