How to Build a Closed‑Loop AIOps System with LLMs, MCP, and DevOps

Why Conversational Ops?

Operations engineers often waste time switching between Jenkins, Prometheus, Grafana, and Kubernetes CLIs, remembering API endpoints and command syntax. New team members need weeks to master this toolchain, and undocumented tacit knowledge makes onboarding painful.

Conversational Ops Concept

The core idea is to use natural language as a unified entry point for all operational tasks. Users simply state what they want—e.g., “list all Jenkins jobs”—and the system translates the request into tool calls and returns a human‑readable result.

Three‑Layer Architecture

LLM Layer : A DeepSeek large language model interprets user intent, plans the required action, and formats the tool response into readable text. It acts as the “operations translator.”

MCP Layer (Model Context Protocol): Defines a standard protocol for the LLM to discover and invoke external tools. It works like a “standard socket” that safely routes calls to any DevOps service.

DevOps Tool Layer : Actual operational tools such as Jenkins, Kubernetes, Prometheus, and Zabbix. In the prototype only Jenkins is integrated, providing job listing, build triggering, and status queries.

The closed‑loop flow is: user utterance → LLM intent parsing → MCP tool dispatch → tool execution → JSON result → LLM formatting → user response.

Technical Architecture

The stack is visualised in the diagram below (image omitted for brevity). The LLM layer runs the DeepSeek model, the MCP layer implements a lightweight server that maps actions to tool APIs, and the DevOps layer exposes REST endpoints (e.g., Jenkins API).

Closed‑Loop Workflow (Listing Jenkins Jobs)

User query : The Streamlit UI captures free‑form text such as “列出所有任务” or “帮我看看现在 Jenkins 上有啥”.

LLM intent parsing : The model receives the conversation history and a system prompt that enumerates supported actions (list_jobs, trigger_build, …) and the required JSON schema. With temperature 0.1 the output is deterministic JSON, e.g., {"action":"list_jobs", "reason":"用户想查看 Jenkins 上现有的全部任务"}.

MCP tool call : The orchestrator maps the parsed intent to a concrete tool. In the prototype the orchestrator itself acts as the MCP router, dispatching to JenkinsService.list_jobs().

Tool execution : JenkinsService.list_jobs() authenticates with Jenkins using username + API token (Basic Auth), calls server.get_all_jobs(folder_depth=3), and builds a JSON payload containing job name, URL, colour status, and timestamp.

Result feedback : The LLM receives the raw JSON, converts colour codes (blue = success, red = failure, blue_anime = building) into a markdown‑style table, and sends it back to the Streamlit chat window.

Prototype UI

The front‑end is a Streamlit app: the left pane holds Jenkins endpoint, username, and API token fields plus a “Test Jenkins Connection” button; the right pane shows the chat history and model status. After entering “列出所有任务”, the assistant returns a table with three jobs and their health states.

Key Implementation Details

Jenkins API Token : Generated from “User → Settings → API Token”. It is used as the password in Basic Auth; the token is shown only once, so it must be saved securely.

MCP Server Design : Each DevOps tool runs its own MCP server exposing a uniform set of actions. Adding a new tool only requires implementing a new server; the LLM does not need code changes.

Prompt Engineering : The system prompt lists every supported action, its parameters, and the exact JSON output format. Iterative refinement was needed to achieve stable, markdown‑free responses.

Security Boundary : Query‑type actions are executed directly, while mutating actions (e.g., triggering builds) require a second‑step user confirmation. Future work includes role‑based access control and audit logging.

Value Proposition

Conversational AIOps lowers the operational barrier for newcomers, speeds up incident investigation by eliminating tool‑hopping, and codifies tacit troubleshooting knowledge into prompts and tool definitions, turning experience into reusable assets.

Future Directions

Planned extensions include adding Kubernetes (pod inspection, scaling), Prometheus (metric queries, alert analysis), and integrating Retrieval‑Augmented Generation (RAG) to feed internal runbooks and historical incidents into the LLM, enabling both execution and diagnostic suggestions.

In summary, the prototype demonstrates that the combination of a cost‑effective LLM, a standardized MCP bridge, and existing DevOps tools can deliver a functional closed‑loop AIOps system, turning routine operations into natural‑language conversations.