Task Decomposition with Multi‑Agent Systems: Boosting Complex AI Workflows

Overview

The thesis "Task Decomposition with Multi‑Agent Systems" (UCB/EECS‑2026‑123) investigates whether increasingly capable foundation models still require explicit task decomposition for complex problems. The author argues that single models excel at end‑to‑end generation but many real tasks involve multiple sub‑problems that benefit from modular planning, execution, verification, and correction.

Why Explicit Visual Structure Matters

Complex image prompts often contain negation, counting, attribute binding, and spatial relations (e.g., "no bananas on the table" or "three cats in an L‑shape"). Pure diffusion models frequently violate these constraints, indicating a need for intermediate, inspectable representations rather than relying solely on richer text embeddings.

Six Core Contributions

1. LLM‑grounded Diffusion (LMD) : The pipeline splits text‑to‑image generation into (1) an LLM that parses the prompt into a structured layout of captioned bounding boxes, background description, and optional negative prompts, and (2) a diffusion controller that generates the image conditioned on this layout. This design requires no retraining of the diffusion model; the LLM acts as a planner.

2. LLM‑grounded Video Diffusion (LVD) : Extends LMD to video by having the LLM produce a Dynamic Scene Layout (DSL) that specifies object bounding‑box trajectories across frames. The video diffusion model then respects these temporal constraints, enabling coherent motion and perspective handling.

3. Self‑correcting LLM‑controlled Diffusion (SLD) : Introduces a closed‑loop where the generated image is examined by an open‑vocabulary detector and an LLM parser. Detected mismatches (missing objects, wrong count, misplaced items) trigger the LLM to output corrective bounding‑box or latent‑space adjustments, which are fed back to the diffusion model.

4. Describe Anything Model (DAM) : Focuses on fine‑grained local description. Users specify a focal region via a prompt; the model combines a local visual backbone with global context through gated cross‑attention, producing detailed captions for the region while preserving overall scene understanding. The authors also release DLC‑Bench for evaluating such local descriptions.

5. Adaptive Parallel Reasoning (APR) : Introduces explicit spawn and join operations that let a language model launch parallel sub‑threads to explore different branches of a reasoning problem (e.g., Countdown). Experiments show APR utilizes additional compute more efficiently than simple Best‑of‑N or serial search, reducing latency while preserving accuracy.

6. ThreadWeaver : Extends APR to strong LLMs by dynamically generating concurrent threads on a standard inference engine. The approach keeps the original model unchanged, using a special trajectory format and request scheduling to simulate fork‑join execution. Benchmarks on mathematical reasoning tasks show comparable accuracy to sequential baselines with reduced token‑level critical‑path latency.

Unified Perspective

The thesis frames multi‑agent systems not as a collection of independent models but as an organized computation graph: LLMs plan, visual models execute, detectors verify, and parallel threads explore. This organization enables explicit intermediate representations, controllable generation, and self‑diagnosis, which are essential for reliable deployment of AI systems.

Future Directions

Open challenges include improving the reliability of intermediate representations, developing credit‑assignment mechanisms to pinpoint failures across agents, jointly optimizing efficiency and robustness, and building adaptive systems that decide when and how to decompose tasks based on difficulty and constraints.

Overall, the work demonstrates that as foundation models grow, the bottleneck shifts from raw capability to effective task structuring and compute orchestration.