Building and Understanding LocalAI Agents: From YAML Config to Source Code

1. Getting Started with a LocalAI Agent

LocalAI lets you define an agent with a simple .yaml file instead of writing Go code. The example creates an “ops‑agent.yaml” that sets the model, temperature, and a system prompt describing a senior Linux administrator.

name: ops-agent
context_size: 4096
# system prompt
parameters:
  model: llama-3-8b-instruct
  temperature: 0.2  # low temperature for precise ops answers
  top_k: 40
  top_p: 0.9

template:
  chat: |
    <|im_start|>system
    You are a senior Linux system administrator. Answer accurately and concisely with command‑line solutions.
    <|im_end|>
    <|im_start|>user
    {{.Input}}
    <|im_end|>
    <|im_start|>assistant

After restarting LocalAI, the agent can be invoked like a normal model via the /v1/chat/completions endpoint:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ops-agent",
    "messages": [{"role":"user","content":"My disk is full, how to troubleshoot?"}]
}'

"You can use df -h to view disk usage, then run du -sh * | sort -rh | head -n 10 to find the largest files."

2. From Chat to Agent: Source‑Code Perspective

The agent handling code lives in core/http/endpoints/localai/agents.go. Registration of routes occurs in core/http/routes/agents.go, exposing /api/agents for listing, creating, and chatting with agents.

func RegisterAgentPoolRoutes(e *echo.Echo, app *application.Application, ...) {
    ag := e.Group("/api/agents", poolReadyMw, agentsMw)
    ag.GET("", localai.ListAgentsEndpoint(app))
    ag.POST("", localai.CreateAgentEndpoint(app))
    ag.POST("/:name/chat", localai.ChatWithAgentEndpoint(app))
}

ChatWithAgentEndpoint processes the request, forwards it to the service layer, and returns a message_id. The system then streams progress and final answer via Server‑Sent Events (SSE), enabling asynchronous tool calls and multi‑step reasoning.

3. State and Memory

Agents are state‑driven. GetAgentStatusEndpoint reports the current reasoning steps and executed actions, while GetAgentObservablesEndpoint provides short‑term memory of environment changes. This design supports multi‑turn conversations and long‑running tasks.

Status – records current thought steps, actions, and results.

Observables – short‑term memory of perceived changes.

3. Advanced: Making Agents Do Work

Beyond the language model, agents can be equipped with tools to read local files, execute database queries, or perform web searches. The next article will cover the Model Context Protocol (MCP) for safe tool integration.