Building Production-Grade AI Agents with AgentCore & Codex: Architecture & Hands-On Guide

Technical Design: AgentCore + Codex Architecture

The article demonstrates how to build an end-to-end Agent application by combining a custom Harness framework (Codex) with AgentCore's capabilities: identity, model management, tools, skills, credentials, observability, and evaluation. The architecture consists of two main components:

agentcore-chat : Interactive intelligent application frontend.

agentcore-runloop : Runtime framework connecting AgentCore infrastructure with the custom Harness (Codex).

1. Overall Architecture

The system uses a layered design where AgentCore provides foundational services (model connections, tool registry, skill storage, identity, credentials, observability) and the Harness (Codex) consumes these via standard protocols. The Runloop acts as an adapter, exposing AgentCore capabilities to Codex through MCP (Model Context Protocol) aggregators.

2. AgentCore-Runloop Internal Execution Architecture

The Runloop manages the agent execution lifecycle: it receives requests, coordinates with AgentCore services, injects configurations (model endpoints, credentials, tool definitions) into the Harness, and handles multimodal file materialization. A diagram illustrates the flow from user input through Runloop to Codex and back.

3. Model Integration

Models are registered in AgentCore, with connection details and credentials injected via environment variables. The SDK queries model connections and passes them to Codex for mounting. This decouples model management from the Harness.

4. Tools

AgentCore provides resource metadata and data-plane credentials. The Harness compiles these into a session-level MCP Tool Registry. Codex connects only to the Runloop MCP Aggregator using standard tools/list and tools/call methods. Actual authentication, human-in-the-loop (HITL), routing, and execution are handled by the Harness.

5. Skills

AgentCore supplies Skill metadata and controlled download endpoints. The Harness handles signed downloads, installation into the workspace, and injection of skill indexes into Codex. Codex progressively reads SKILL.md and attachments only when tasks match. Skills are versioned and persisted in isolated NAS/AgenticFS directories, mapped read-only to a unified .agents/skills/ logical path, decoupling physical storage, execution environment, and internal references.

6. Storage

Agents generate and consume files (uploads, repos, intermediate results, artifacts). AgentCore offers two storage types:

NAS (AgenticFS) : POSIX filesystem semantics, supports directory structure, random read/write, atomic rename, and many small files. Ideal as persistent session workspace for code checkout, Git ops, dependency extraction, file edits, and multi-turn reuse.

OSS : Object storage for large objects and independent-lifecycle files (user uploads, read-only inputs, model data, final artifacts, cross-system delivery). Not suited for random modifications or directory ops; not used as agent working directory.

Recommended pattern: OSS object exchange layer + AgenticFS session workspace . AgenticFS handles process data and persistence; OSS serves final results to users. User-level file isolation is already implemented; future sandbox-based isolation planned.

7. Identity & Security

Agent identity is a critical, often underestimated capability for production trust. Each Agent gets an Alibaba Cloud AgentIdentity workload identity, linkable to trusted identity providers and RAM roles, enabling secure cloud resource management with Alibaba Cloud as trust anchor.

Identity comprises three contexts:

Business Identity : uid, tenant, cloud account — represents who the task is for.

Execution Identity : conversationId and runId — for session isolation and audit.

Resource Identity : Allowed tools, skills, sandboxes, and sub-agents for this run.

User identity and resource ownership are verified by a trusted gateway. uid is for isolation/tracing only, not an authentication credential.

Temporary Credentials

Agents access cloud resources via short-lived, least-privilege STS credentials issued on-demand by a trusted service, bound to account, region, resource scope, permissions, and TTL. Credentials stay in Runloop control plane; Runloop signs requests on behalf of the model. Model, Codex, and workers never see secrets. Credentials are not written to prompts, event history, logs, or workspace. Expired credentials are re-issued; no fallback to permanent AccessKeys.

The Runloop framework includes a built-in workload credential service, used for granting minimal OSS access to view agent-generated files.

8. Observability & Evaluation

Runloop integrates with AgentLoop (Alibaba Cloud's observability platform) via OpenTelemetry instrumentation configured through environment variables. Applications only configure AgentLoop reporting parameters; no manual span creation in business code. The article lists all required environment variables (e.g., RUNLOOP_AGENTLOOP_ENDPOINT, RUNLOOP_AGENTLOOP_LICENSE_KEY, RUNLOOP_AGENTLOOP_PROJECT, etc.) with descriptions and acquisition steps via AgentLoop console.

9. Multimodal Support

Current implementation uses a file-driven approach: text passes via AG-UI directly; images/documents are materialized as files in the agent workspace, then read via controlled tools; generated artifacts (PPT, images, docs) are published as files for frontend preview.

Implementation Flow

User selects/pastes file in frontend.

Frontend requests short-lived upload URL via /api/uploads, uploads directly to OSS.

Backend validates file size, type, ETag, and ownership (user/session).

Before agent invocation, deployment adapter downloads attachment to isolated workspace (e.g., /workspace/uploads/<uploadId>/file.png).

Runloop converts attachment to user.attachment event, writes file path into agent context.

Agent calls view_image, file reading, or document processing tools based on file type.

This decouples Runloop from storage: Runloop never accesses OSS, issues upload credentials, or holds storage keys; it only consumes materialized file paths. The approach is tool-augmented file multimodality , not native model multimodal content blocks. Capabilities: Runloop compresses attachments to file references; image understanding via view_image; document understanding via file reading/parsing tools; output via publish_artifact; AG-UI returns only text, tool events, and controlled file references (no binaries). This works across different Agent CLIs and models with consistent file handling, security isolation, and artifact delivery.

Practical Tutorial: Building & Deploying a Production Agent

Prerequisites: Cloud Resources

Enable Alibaba Cloud services: AgentCore, AgentLoop, SLS, CMS, AgentIdentity, ACR (enterprise instance), RAM, FC, OSS, NAS (apply for AgenticFS whitelist, then create filesystem).

Build & Deploy

Build Image : Compile from agentcore-runloop source, push to ACR.

Configure AgentCore Resources : Set up model connections, Skills, MCPs, credentials, memory, knowledge bases. Record names/versions (knowledge bases and memory are MCP-based).

Model Configuration

Create model connection in AgentCore console (e.g., hx-model-connection with model qwen3.8-max).

Skill Configuration

Add Skills (e.g., [email protected], remotion-best-practices) with versions.

Credential Configuration (Multimodal)

Configure Bailian (Alibaba Cloud Model Studio) API keys for vision ( qwen-vl-max) and generation ( qwen-image-plus). Set via environment variables AGENTCORE_IMAGE_VISION_MODEL and AGENTCORE_IMAGE_GENERATION_MODEL. Credential key is DASHBOARD_API_KEY, scoped to only those models to prevent privilege escalation.

Knowledge Base Configuration

Use Bailian knowledge base; pre-create and grant permissions. Requires dual authorization: execution role needs AliyunBailianDataFullAccess; Bailian console must grant data operation rights to that role.

Memory Configuration

Memory is consumed via MCP. Create memory in AgentCore, enable MCP service.

Assemble Agent Service

Choose "High-Code" mode, select uploaded agentcore-runloop image.

Set startup command: node /app/agentcore-runloop/scripts/start.mjs.

Configure environment variables (table below).

Key Environment Variables

AGENTCORE_MODEL_CONNECTION

: Model connection name (e.g., hx-model-connection) AGENTCORE_MODEL_NAME: Model name (e.g., qwen3.8-max) AGENTCORE_SKILLS: Comma-separated skills (e.g., [email protected],remotion-best-practices) AGENTCORE_SYSTEM_PROMPT: System prompt (e.g., You are Xiao Hong, a helpful assistant) AGENTCORE_MCP_SERVERS: MCP services (e.g., tool-mcp,bailian-tool-9lhdp,hx-memory) AGENTCORE_WORKDIR: Persistent workspace path on AgenticFS (e.g., /mnt/agenticfs/shared/users) AGENTCORE_ARTIFACT_ROOT: Public artifact root on OSS (e.g., /mnt/oss/data/users)

Agent Protocol & Storage Config

Configure AG-UI protocol, OSS bucket, and NAS filesystem in AgentCore console. Test via console debug before proceeding.

Observability Setup

Inject AgentLoop environment variables (listed in article) obtained from AgentLoop console → Manual Instrumentation → Node.js. Key variables: RUNLOOP_AGENTLOOP_ENDPOINT (HTTPS OTLP trace endpoint), RUNLOOP_AGENTLOOP_LICENSE_KEY (store in Secret), RUNLOOP_AGENTLOOP_PROJECT, RUNLOOP_AGENTLOOP_WORKSPACE, RUNLOOP_AGENTLOOP_SERVICE_NAME, RUNLOOP_AGENTLOOP_SERVICE_VERSION (use image tag or Git SHA), RUNLOOP_AGENTLOOP_ENVIRONMENT (staging/production), RUNLOOP_AGENTLOOP_CAPTURE_CONTENT (collect prompts, responses, tool params/results), RUNLOOP_AGENTLOOP_CONFIG_SOURCE ( env or agentcore), RUNLOOP_AGENTLOOP_CREDENTIAL_NAME (AgentCore credential name when using AgentCore config source).

Demo Results

The article demonstrates the deployed agent via agentcore-chat frontend (clone GitHub repo https://github.com/ai-agentcore/agentcore-chat to run locally). Capabilities shown:

Basic: View agent persona and included skills/tools.

Tool Calling: Test MCP tools (screenshots show tool invocation and results).

Skill Invocation: Trigger PPTX skill, generate artifact, preview file.

Knowledge Base: Import data (China Broadcasting terminal models), test precise retrieval.

Memory: Demonstrate memory read/query.

Multimodal: Upload screenshot of AgentCore website, agent analyzes image.

Observability: AgentLoop dashboards show basic metrics, entity details, trace waterfall, and agent loop details.

Performance Metrics

Custom Codex typical TTFT (Time To First Token) ~1.7s, P95 ~1.95s. Gateway starts SSE response ~0.18s; main latency in model first-token generation. 20 samples showed no stability issues or severe long tails.

Source Code & Links

AgentCore-Runloop : https://github.com/ai-agentcore/agentcore-runloop AgentCore-Chat :

https://github.com/ai-agentcore/agentcore-chat