MASCOT: Two-Stage Training for Diverse Multi-Agent Conversations
Researchers from Georgia Tech and MIT propose MASCOT, a two-stage training framework that teaches AI agents distinct personas and a director agent to coordinate turn-taking, enabling diverse, coherent multi-agent conversations for emotional support and meeting scenarios.
Researchers from Georgia Tech and MIT introduce MASCOT (Multi-Agent Socio-Collaborative Companion Systems), a two-stage training framework designed to make multi-agent group chats more diverse and collaborative. The paper addresses a core problem: when multiple AI agents converse with a user, they often fall into persona collapse — repeating similar supportive statements instead of providing complementary perspectives.
Two-Stage Training Framework
Stage 1: Persona-Aware Behavioral Alignment
Each agent learns its designated role through a preference-based reward model. For a given situation and persona, the system generates multiple candidate responses and uses a strong model to create preference pairs that clearly distinguish high-quality, persona-consistent responses from generic ones. A reward model is trained on these pairs to judge whether a response fits the persona and the current context. The speaking models are then fine-tuned with GRPO (Group Relative Policy Optimization) using the reward scores, making persona-appropriate responses more probable.
In the emotional-support experiments, three personas are defined:
Alice (Listener) : focuses on empathetic listening and non-judgmental validation.
Bob (Encourager) : highlights positive experiences and strengths, offering concrete encouragement.
Cathy (Analyst/Reflection Guide) : asks probing questions and encourages perspective-taking to help the user explore causes and next steps.
Stage 2: Director Agent for Turn Coordination
Even with well-trained personas, the conversation can become disjointed if agents speak at the wrong time. MASCOT freezes the speaking models and trains a Director Agent that reads the dialogue history, selects the next speaker, and provides a response strategy (e.g., continue listening, guide reflection, discuss action). The Director is trained with GRPO using a group-level reward signal derived from simulated user continuations: the simulated user engages with the generated response, and the resulting extended dialogue is evaluated for coherence, diversity, and effective contribution. This teaches the Director which speaker selections lead to better overall interactions.
Experimental Setup
MASCOT is evaluated on two domains requiring multi-perspective collaboration:
Emotional Support : Empathetic Dialogues (ED) dataset, plus out-of-distribution generalization on ESConv.
Work Collaboration : QMSum meeting summarization dataset (academic, committee, product meeting subsets).
Baselines include prompt-based methods (Few-Shot, Few-Shot + CoT), supervised fine-tuning, inference-time strategies, and dedicated multi-agent frameworks (e.g., MultiAgentESC). Evaluation measures both individual metrics (empathy quality, persona consistency, response contribution) and group metrics (coherence, novelty, relevance, participation). Automatic scoring uses a model judge on a 0–100 scale, cross-validated with human blind evaluation and judges from different model families.
Key Results
Emotional Support (ED Dataset)
Persona Consistency : MASCOT achieves 80.0 in positive contexts vs. strongest baseline 65.9 (+14.1).
Individual Overall Scores : MASCOT outperforms baselines across positive, negative, and neutral contexts (e.g., 76.9 vs. 67.6 in positive). Negative contexts remain more challenging.
Group Interaction (Neutral Context) : MASCOT scores highest on all four group metrics — persona fidelity, novelty, relevance, participation — with an overall 57.3 vs. Few-Shot + CoT's 51.9 (+5.4).
Meeting Collaboration (QMSum)
MASCOT leads on group interaction overall scores across all three meeting subsets, showing the framework transfers to task-oriented discussions where roles include recording, decision tracking, action proposals, and gap checking.
Ablation Study
MASCOT-P (without persona alignment): larger drops in individual empathy quality and persona consistency.
MASCOT-C (without collaboration optimization, random speaker order): larger drops in group coherence and participation.
This confirms the two stages address distinct capabilities: persona alignment stabilizes individual behavior; collaboration optimization teaches effective turn-taking.
Human Evaluation
40 annotators (ages 19–73, mean 42.9) performed blind pairwise comparisons. MASCOT was preferred over Few-Shot CoT in 69.0% of 200 judgments and over MultiAgentESC in 68.5% of 200 judgments.
A second study with 20 participants comparing MASCOT vs. Few-Shot CoT on 100 dialogues (200 judgments) showed a 75.5% preference for MASCOT. Annotators also rated usefulness, emotional appropriateness, and perceived safety.
Personality Adaptation (Big Five Simulation)
Using OCEAN traits to simulate 10 user profiles (High/Low on each dimension, others neutral) across 50 ED situations (500 dialogues):
High Openness → persona consistency 79.0.
High Agreeableness → overall score +1.9 vs. Low Agreeableness.
High Extraversion → social contribution +2.5 while maintaining persona consistency.
High Conscientiousness → overall score 71.5 vs. Low 74.5 (more challenging).
High/Low Neuroticism → similar overall scores (70.7 vs. 70.8).
Results suggest future systems need personalized role composition and coordination strategies.
Limitations & Future Work
Current evidence comes from English dialogues and controlled evaluations. Long-term interaction, cross-cultural generalization, and adaptation to evolving user goals and emotions remain open questions.
Resources
Paper: MASCOT: Multi-Agent Socio-Collaborative Companion Systems
Authors: Yiyang Wang, Yiqiao Jin, Alex Cabral, Josiah Hester
arXiv: https://arxiv.org/abs/2601.14230 Project page: https://hello-diana.github.io/MASCOT/ Code:
https://github.com/hello-diana/MASCOTSigned-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
