How BiCNet Enables Multi‑Agent Cooperation in StarCraft Battles

Introduction

Alibaba's Cognitive Computing Lab and UCL collaborated to study multi‑agent cooperation using the micro‑battle scenarios of the real‑time strategy game StarCraft 1. The goal is to develop collaborative AI capable of solving problems that individual agents cannot handle.

BiCNet Architecture

BiCNet (Bidirectional Coordination Network) extends the actor‑critic framework to a vectorized form where each dimension corresponds to an agent. It consists of an actor (policy) network and a critic (Q‑value) network, both built on bidirectional recurrent neural networks (RNNs). Parameter sharing across agents keeps the model size independent of the number of agents, while bidirectional connections enable effective communication.

The actor network allows each agent to maintain its own internal state while sharing information with teammates. The critic network receives the joint state‑action input and outputs local Q‑values, which are combined to estimate a global return.

Learning Cooperative Strategies

After training, BiCNet automatically discovers five distinct cooperative behaviors:

Collision‑free coordinated movement

Attack‑and‑retreat tactics

Cover attacks

Focused fire without over‑shooting

Heterogeneous agent cooperation

Examples include three Marines moving without colliding against a Super Zergling, coordinated attacks and retreats, and complex cover‑attack maneuvers where one unit draws fire while others strike.

In cover‑attack scenarios, a Dragoon unit retreats while a teammate attacks the enemy, then roles reverse, creating a continuous protective loop that minimizes casualties.

Focused‑fire strategies learn to concentrate attacks on one or two enemies while distributing other agents to cover additional targets, demonstrating dynamic grouping based on unit positions.

Heterogeneous cooperation is shown with Dropships and Tanks jointly defending against an Ultralisk, where Dropships transport and protect Tanks while the Tanks engage the enemy.

Performance Comparison

BiCNet outperforms existing state‑of‑the‑art methods (e.g., CommNet) across a range of battle configurations, including varying numbers of Marines versus Zerglings and mixed‑unit engagements. Results show higher win rates and more efficient coordination.

Conclusion

The bidirectional coordination network provides a scalable deep multi‑agent reinforcement learning framework that learns effective cooperative policies end‑to‑end. Experiments demonstrate its ability to acquire diverse strategies in StarCraft micro‑battles, suggesting promising applications in e‑commerce, gaming, healthcare, and other domains requiring coordinated AI decision‑making.