Can Dual-Agent AI Transform Web Video Editing? Inside VibeCut’s Architecture

Introduction

To address the complexity of professional video editing software and the creative limits of template‑based tools, this paper presents VibeCut, an intelligent editing system for the WebCut platform. VibeCut aims to bridge fully manual and fully automatic editing by offering an efficient, user‑friendly, and personalized editing paradigm.

Core Architecture

The system is built on a novel Orchestrator‑Executor (planner‑executor) dual‑agent architecture. The Orchestrator deeply understands natural‑language user intent and creates a high‑level task plan, while the Executor focuses on invoking specific tools to perform concrete actions. Both agents share a structured Shared Context that serves as the single source of truth for commands and state, decoupling planning from execution and reducing the cognitive load on large language models (LLMs).

Background and Motivation

Traditional professional editors (e.g., Adobe Premiere, Final Cut) are powerful but have steep learning curves, whereas lightweight online editors are easy to use but often produce homogeneous content. Recent advances in large language models, function calling, and multi‑agent systems provide new opportunities to combine the depth of professional tools with the convenience of online platforms.

Related Work

We review existing AI‑enhanced video editing solutions, including OpenAI’s Manager pattern, Anthropic’s Orchestrator‑Workers and Evaluator‑Optimizer modes, and Cognition AI’s single long‑running agent approach. These works highlight the trade‑offs between modular multi‑agent coordination and context management.

Multi‑Agent Design Principles

Separate task planning from tool execution to lower LLM burden.

Maintain a centralized, structured Shared Context that is human‑readable and UI‑friendly.

Operate directly on the draft (timeline) rather than through complex UI interactions.

VibeCut Implementation

VibeCut integrates three primary tool categories:

UI Interaction Tools : handle user prompts, approvals, and feedback.

Resource Retrieval Tools : query assets, perform semantic video understanding, and retrieve relevant media.

Editing Tools : manipulate the draft timeline, apply cuts, subtitles, transitions, and other edits.

Both agents communicate via the Shared Context. The Orchestrator generates a plan, updates task status after each tool execution, and re‑plans when failures occur. The Executor selects the appropriate tool based on the current sub‑task and executes it.

Experiments

We evaluated VibeCut on three representative editing scenarios: custom subtitle styling, adaptive subtitle color based on visual content, and semantic video trimming. Metrics include token consumption, processing time, output quality, and failure rate. Each scenario was run three times and averaged.

Results

The system successfully transformed vague natural‑language requests into concrete, controllable editing actions, achieving comparable quality to manual editing while significantly reducing user effort. However, high context length requirements and ambiguous user intents sometimes caused plan deviations.

Ablation Study

We compared different LLM back‑ends (deepseek‑v3, deepseek‑r1, qwen3‑8b) for both Orchestrator and Executor. Larger models produced more reliable planning and tool selection, while smaller models struggled with structured output generation and state evaluation.

Case Study: Text‑to‑Video Templates

VibeCut was extended with three new tools—storyboard script generation, character image synthesis, and storyboard image creation—to handle text‑driven template video production. Using qwen‑image models, the system generated coherent storyboards and assembled them into a final video draft, demonstrating the flexibility of the shared‑context architecture.

Conclusion

The dual‑agent, shared‑context design proves feasible for web‑based intelligent video editing, offering a scalable path between fully manual and fully automatic workflows. Key insights include the importance of decoupling planning from execution, the central role of a structured context, and the dependence of overall capability on the richness of underlying tools.

Future Work

Model and performance optimization: fine‑tune lightweight models on a curated video‑editing dataset to reduce token cost and latency.

Expand multimodal capabilities: incorporate voice commands, audio emotion analysis, and visual style transfer.

Enrich toolset: add intelligent music generation, B‑roll matching, audio denoising, and motion‑tracking effects.

Persist user preferences and develop a benchmark suite for AI‑driven video editing tasks.

References

OpenAI, "A Practical Guide to Building Agents" (PDF).

Anthropic, "Building Effective Agents".

Cognition AI, "Don't Build Multi‑Agents".

Manus, "Context Engineering for AI Agents".

ArXiv, "Efficient Fine‑Tuning of Small LLMs for Domain Tasks".