How Reinforcement Learning Powers AI Bots in ‘Barbarian Battle 2’

On March 24, 2023, NetEase Zhiji partnered with Dianhun Network to showcase at GDC 35 the application of reinforcement learning‑based AI robots in the game Barbarian Battle 2 . The collaboration covered AI integration, training, deployment and iteration.

Key components:

Integration scheme: design of gameplay experience and access methods.

Robot training: SAR solution design, network architecture, large‑scale distributed training.

Robot deployment: multi‑difficulty and multi‑style AI interfaces, dynamic difficulty adjustment, private deployment support.

Robot iteration: model fine‑tuning or structural adjustments after major version updates.

Traditional rule‑based bots (FSM, behavior trees) cannot capture the complexity of human players, leading to predictable and unsatisfying behavior. Reinforcement learning (RL) trains an agent by interacting with the game environment, receiving rewards, and learning optimal policies, resulting in more human‑like and challenging AI.

To reduce training cost, NetEase Fuxi co‑developed the distributed RL framework RLEase , consisting of Workers that collect state‑action‑reward data from the game and Learners that train the model. Supporting modules include Stat for metrics, and Model Manager for model storage and opponent selection based on win‑rate.

The AI model processes multi‑head inputs (character, team, opponent, boss, map states) and outputs actions and value estimates (the latter used only during training). Feature engineering such as sector‑based mushroom counting, perception‑range limiting, and embedding‑based one‑hot replacement helps manage the large state space.

Curriculum learning gradually introduces more heroes and equipment, while reward shaping encourages desired behaviors (e.g., focusing fire on the boss). Three robot styles—cautious, aggressive, supportive—are created by adjusting reward weights.

AIBridge middleware abstracts the AI service, exposing simple APIs (GameStart, tick, GameEnd) so developers can integrate AI bots without deep RL knowledge.

Debugging includes real‑time data statistics, video replay analysis, and handling bugs that corrupt training data.

Overall, the project demonstrates that RL‑based AI bots can enhance player experience, and the presented solutions (RLEase, AIBridge, feature engineering, curriculum learning) address the challenges of cost, scalability, and integration.