Deploy AI Agents Seamlessly with AgentScope and Knative Serverless

Enterprises facing the AI wave need a flexible, efficient way to deploy intelligent agents; the integration of AgentScope with Knative provides a serverless container platform that handles version management, automatic scaling (including scale‑to‑zero), and traffic routing.

What is AgentScope?

AgentScope Runtime is a comprehensive agent runtime framework that tackles two main challenges: high‑performance agent deployment and sandboxed execution. It includes built‑in services for short‑term memory, state persistence, and a secure sandbox infrastructure, offering full observability and developer‑friendly deployment.

Key Features of AgentScope

Deployment Infrastructure : Built‑in services for agent state, session history, long‑term memory, and sandbox lifecycle control.

Framework‑agnostic : Works with any popular open‑source agent framework or custom implementation.

Developer‑friendly : Provides the AgentApp utility for easy deployment and extensive customization options.

Observability : Comprehensive tracing and monitoring of runtime operations.

Sandboxed Tool Execution : Isolated sandbox ensures safe tool execution without affecting the host system.

Out‑of‑the‑Box & One‑Click Adaptation : Rich set of ready‑to‑use tools and adapters for rapid framework integration.

Typical Deployment Pain Points

Frequent Version Iterations : Rapid releases and A/B testing demand fast roll‑backs, which traditional pipelines struggle to support.

Elasticity & Stability : Need to auto‑scale resources up on demand and scale‑to‑zero when idle, while maintaining service stability.

Runtime Security : Agents must run in isolated environments to prevent prompt‑injection attacks or unauthorized data access.

Vendor Lock‑in : Preference for open, cloud‑agnostic technologies over proprietary stacks.

Why Knative?

Multi‑Version Management : Supports simultaneous versions for A/B testing via label routing and traffic splitting.

Request‑Based Auto‑Scaling : Configurable concurrency or RPS metrics; automatically scales pods to zero when no traffic.

ACS Compute Support : Each pod runs in a secure sandbox container, offering isolation and large‑scale elasticity.

Standardized Docker Images : Runs on any Knative‑compatible environment (public cloud, private cloud, edge), avoiding vendor lock‑in.

Quick Start

1. Install from source

# Clone the repository
git clone -b main https://github.com/agentscope-ai/agentscope-runtime.git
cd agentscope-runtime
# Install core dependencies
pip install -e .

2. Create project structure

my-agent-project/
├── app_agent.py      # Main agent file
├── requirements.txt  # Optional Python dependencies
└── .env              # Optional environment variables

3. Write agent code (example)

# -*- coding: utf-8 -*-
import os
from agentscope.agent import ReActAgent
from agentscope.formatter import DashScopeChatFormatter
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import stream_printing_messages
from agentscope.tool import Toolkit, execute_python_code
from agentscope_runtime.adapters.agentscope.memory import (
    AgentScopeSessionHistoryMemory,
)
from agentscope_runtime.engine.app import AgentApp
from agentscope_runtime.engine.schemas.agent_schemas import AgentRequest
from agentscope_runtime.engine.services.agent_state import (
    InMemoryStateService,
)
from agentscope_runtime.engine.services.session_history import (
    InMemorySessionHistoryService,
)

agent_app = AgentApp(
    app_name="MyAssistant",
    app_description="A helpful assistant agent",
)

@agent_app.init
async def init_func(self):
    """Initialize services."""
    self.state_service = InMemoryStateService()
    self.session_service = InMemorySessionHistoryService()
    await self.state_service.start()
    await self.session_service.start()

@agent_app.shutdown
async def shutdown_func(self):
    """Cleanup services."""
    await self.state_service.stop()
    await self.session_service.stop()

@agent_app.query(framework="agentscope")
async def query_func(self, msgs, request: AgentRequest = None, **kwargs):
    """Process user queries."""
    session_id = request.session_id
    user_id = request.user_id
    state = await self.state_service.export_state(session_id=session_id, user_id=user_id)
    toolkit = Toolkit()
    toolkit.register_tool_function(execute_python_code)
    agent = ReActAgent(
        name="MyAssistant",
        model=DashScopeChatModel(
            "qwen-turbo",
            api_key=os.getenv("DASHSCOPE_API_KEY"),
            enable_thinking=True,
            stream=True,
        ),
        sys_prompt="You're a helpful assistant.",
        toolkit=toolkit,
        memory=AgentScopeSessionHistoryMemory(
            service=self.session_service,
            session_id=session_id,
            user_id=user_id,
        ),
        formatter=DashScopeChatFormatter(),
    )
    agent.set_console_output_enabled(False)
    if state:
        agent.load_state_dict(state)
    async for msg, last in stream_printing_messages(
        agents=[agent],
        coroutine_task=agent(msgs),
    ):
        yield msg, last
    state = agent.state_dict()
    await self.state_service.save_state(user_id=user_id, session_id=session_id, state=state)

if __name__ == "__main__":
    agent_app.run()

4. Pre‑deployment requirements

# Ensure Docker is running
docker --version
# Verify Kubernetes access
kubectl cluster-info
# Log in to your image registry (example: Alibaba Cloud)
docker login your-registry
# Check that Knative Serving is installed
kubectl auth can-i create ksvc

5. Deploy the agent to Knative

agentscope deploy knative app_agent.py \
  --image-name agent_app \
  --env DASHSCOPE_API_KEY=sk-xxx \
  --image-tag linux-amd64-4 \
  --registry-url registry.cn-hangzhou.aliyuncs.com \
  --base-image registry.cn-hangzhou.aliyuncs.com/knative-sample/python:3.10-slim-bookworm \
  --registry-namespace knative-sample \
  --push

The command returns a deployment ID, resource name, and a URL such as http://agent-9e15b9a3.agentscope-runtime.example.com.

6. Test the deployed agent

curl -i -X POST "http://115.29.xxx.xxx:80/process" \
  -H "Content-Type: application/json" \
  -H "Host: agent-9e15b9a3.agentscope-runtime.example.com" \
  -d '{
    "input": [{"role": "user", "content": [{"type": "text", "text": "Hello, how are you?"}]}],
    "session_id": "123"
}'

The response contains a JSON payload with the assistant’s reply.

Observations

When there is no traffic, Knative automatically scales the agent’s pod down to zero, as shown in the accompanying screenshots.

Conclusion

Combining Knative with AgentScope enables rapid development, building, and serverless deployment of AI agent applications, improving service capabilities and resource utilization. Leveraging ACS compute adds default security isolation and large‑scale elasticity, accelerating innovation cycles for AI agents.