One-for-All Multi-Agent Collaboration: Adaptive Cross-Task Topology Design

Problem and motivation

Large language model‑driven multi‑agent systems (MAS) solve complex tasks by assigning distinct roles to agents and coordinating them. A central open question is how agents should collaborate—who speaks first, who passes information, and which experts to involve. Existing automatic topology design methods follow a one‑model‑for‑one‑dataset (one‑for‑one) paradigm, training a separate model for each task domain. This paradigm incurs high maintenance cost, cannot handle cross‑domain user requests, and fails to reuse shared collaboration knowledge.

OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models
Authors: Shiyuan Li, Yixin Liu, Yu Zheng, Mei Li, Quoc Viet Hung Nguyen, Shirui Pan
https://dl.acm.org/doi/abs/10.1145/3774904.3792537
https://github.com/Shiy-Li/OFA-MAS

Why a good topology matters

A MAS’s capability depends not only on individual agents but also on their communication structure. A well‑designed topology lets the right expert participate at the right moment, while a poor topology causes redundancy, error propagation, or inefficient collaboration.

Early approaches used manually designed structures such as Chain, Tree, or Debate, which are simple but inflexible. Recent graph‑learning methods (AgentDropout, G‑Designer, ARG‑Designer) still adopt the one‑for‑one paradigm, training separate models for MMLU, GSM8K, HumanEval, etc. This leads to three practical problems:

Domain assumptions break in reality —real user requests are cross‑domain and unpredictable, making pre‑classification infeasible.

High maintenance cost —adding a new domain requires new data collection, training, and tuning.

Lost cross‑domain knowledge —many tasks share similar collaboration patterns (e.g., "analyzer → checker → solver").

OFA‑MAS aims to train a universal topology designer that learns reusable collaboration patterns from multiple domains and generates a suitable MAS graph for any input query.

Conditional graph generation

OFA‑MAS formulates topology design as a conditional graph generation problem: given a user query and a pool of generic roles, the model directly generates a collaboration graph where nodes are agent roles and edges denote information flow. The autoregressive generation proceeds in three steps:

Select the next agent role based on the current task and the partially generated graph.

Predict communication links for the new role to receive information from existing agents.

Iteratively expand the topology by repeating the "select role – add edges" loop until the graph is complete.

This approach naturally supports one‑for‑all scenarios, allowing each task to produce graphs of varying size, role composition, and connectivity without fixed templates.

Task‑aware graph state encoder (TAGSE)

Pure autoregression is insufficient because the same partial graph can imply different next steps for different tasks (e.g., code generation needs a Reviewer and Debugger, while math reasoning needs a Solver and Verifier). TAGSE injects task semantics into the graph encoding: it encodes the query and role descriptions with a pretrained sentence‑vector model, then applies a context‑gating mechanism to filter out information irrelevant to the current task. Consequently, message passing in the graph is guided by the query, enabling the model to select truly useful structure.

Mixture‑of‑Experts (MoE) graph generation module

Cross‑domain MAS topologies have no single optimal design. OFA‑MAS incorporates an MoE module containing multiple expert networks; a gating network dynamically activates a subset of experts based on task semantics. The MoE module handles:

Node generation : predicting the next agent role.

Edge generation : predicting connections between the new agent and existing agents.

This enables a single model to learn diverse collaboration strategies, activating different expert combinations for different tasks while preserving shared knowledge.

Three‑stage training curriculum

Training a one‑for‑all topology designer is challenging because high‑quality "task‑optimal topology" supervision is expensive. OFA‑MAS adopts a curriculum of increasing difficulty:

Unconditional graph pre‑training —learn basic graph syntax using classic topologies (Chain, Star, Fully‑Connected).

LLM‑driven conditional pre‑training —use a large language model as a "design agent" to synthesize large‑scale "task query – MAS configuration" pairs, teaching the model the correspondence between task semantics and graph structure.

Supervised fine‑tuning —fine‑tune on a small set of high‑quality topologies verified by actual MAS execution on real benchmarks, making the model sensitive to downstream performance.

This curriculum first teaches general graph structure, then aligns it with task semantics, and finally calibrates generation quality with real execution results.

Experimental evaluation

Evaluation covers six representative benchmarks—MMLU, GSM8K, AQuA, MultiArith, SVAMP, and HumanEval—and additionally tests out‑of‑distribution (OOD) generalization on the GAIA benchmark. Baselines include single‑agent CoT, Self‑Consistency, fixed MAS topologies, Debate, and one‑for‑one graph‑learning methods (AgentPrune, AgentDropout, G‑Designer, EIB‑LEARNER).

Overall performance : OFA‑MAS achieves the highest average success rate of 93.02% across the six benchmarks, surpassing all baselines. Using only the first two pre‑training stages (no supervised fine‑tuning) it still reaches 92.15% , beating the strongest baseline EIB‑LEARNER.

OOD generalization : On GAIA tasks unseen during training, OFA‑MAS attains the highest average accuracy and excels on Level‑1 tasks, whereas one‑for‑one methods degrade sharply, sometimes performing worse than a simple Chain topology.

Ablation studies : Removing TAGSE, disabling the MoE module, or omitting any of the three curriculum stages all cause performance drops, confirming that each component is essential for one‑for‑all topology generation.

Robustness test : Simulating malicious agents reduces OFA‑MAS performance by only ~2.2%, far less than competing methods, indicating resilience to unreliable agents.

Case analyses : For HumanEval the generated graph follows a sequential review structure (e.g., Reviewer → Debugger → Solver); for GSM8K the graph combines a math‑solver, program‑assistant, and checker, demonstrating dynamic role selection from a shared pool.

MoE visualization : Tasks from the same domain activate similar expert subsets, while distinct domains (code generation vs. math reasoning) trigger different expert groups, illustrating expert specialization.

Efficiency analysis : Token‑consumption comparisons on MMLU and GSM8K show that OFA‑MAS achieves superior accuracy with competitive inference cost, proving that performance gains stem from more appropriate information‑flow structures rather than simply enlarging the collaboration size.