LLMOps: Building a Prompt‑Driven Engine for AI Operations

The presentation introduces LLMOps, the evolution from AIOps to large‑language‑model‑based operations, emphasizing the need for high‑quality prompts (prompt application) and prompt learning to bridge human intent with model capabilities.

Two major challenges are identified: (1) traditional AIOps algorithms rely on extensive labeled data and lack interpretability, and (2) prompt‑learning data quality is unstable, leading to performance degradation.

To address these, the authors propose the LogPrompt engine, a prompt‑application framework for log analysis that leverages Chain‑of‑Thought (CoT) prompting, implicit and explicit CoT, self‑prompt generation, in‑context examples, and format control to improve both log parsing and anomaly detection, especially in online scenarios with scarce training data.

Extensive experiments on the LogHub dataset using ChatGPT’s API demonstrate that LogPrompt achieves superior zero‑shot performance and better interpretability compared to baseline methods, with higher usefulness and readability scores from expert evaluations.

For prompt learning, a data‑flywheel is built: open‑source datasets (e.g., Alpaca) are generalized, filtered, and refined through human‑in‑the‑loop quality assessment, then used to train a CoachLM that automatically optimizes prompt data, substantially improving data quality (e.g., increasing high‑score samples from 17% to 78%).

Models trained on the optimized data achieve state‑of‑the‑art results on open benchmarks, surpassing existing open‑source LLMs such as Vicuna‑13B, and the approach is extended to small deployable models (e.g., Vicuna‑13B, Vicuna‑7B) showing promising performance despite limited parameters.

The authors envision future work on multi‑modal LLMOps, expanding prompt application and learning to text, image, and speech modalities, and developing lightweight, deployable LLMs for robust, real‑world AI operations.