Game AI Exploration – From AlphaGo to MOBA Games

The talk titled “Game AI Exploration – From AlphaGo to MOBA Games” was delivered by Tencent AI senior researcher Wang Liang at a YunJia community offline salon.

The presentation was divided into four parts: (1) What is game AI and why it matters; (2) Industry and academic methods for game AI and current progress; (3) Research and experiments on a proprietary MOBA game; (4) Requirements for a development environment that supports deep‑learning‑based game AI.

Game AI aims to improve user experience and increase player activity. For game developers it often relies on handcrafted rules, while academic research seeks to maximize win probability using learning‑based methods.

Common uses of game AI include human‑vs‑AI matches, balance testing, and map exploration. A typical architecture consists of perception, decision, and navigation systems that operate in a loop.

Game AI techniques are usually classified into three categories: behavior trees, finite‑state machines, and influence maps (clear logic but rigid); search methods such as genetic algorithms and Monte‑Carlo Tree Search; and learning‑based methods including supervised learning and reinforcement learning.

Supervised learning requires labeled state‑action pairs; defining optimal actions for complex games is difficult. Reinforcement learning replaces explicit labels with reward signals (e.g., win = 10, kill = 1, death = ‑5) and faces challenges such as exploration‑exploitation trade‑offs and reward delay in long‑horizon games.

AlphaGo sparked a wave of game‑AI research. Its algorithm combines supervised learning, reinforcement learning, and Monte‑Carlo Tree Search, and it required massive offline training resources. DeepMind and Blizzard have built an open platform with simulators that are essential for data collection and AI control. Dota 2 AI, trained with >12 W CPU cores and 256 GPUs, can defeat over 90 % of human players under constrained hero selections.

Research on MOBA AI for “Honor of Kings” (王者荣耀) faces several challenges: a vastly larger action space (≈10^5 vs 10^4 for Go), multi‑agent 5v5 scenarios, partial observability, non‑deterministic skill execution, and complex hero skill sets.

Key problems include target selection, hero‑specific skill usage, and knowledge representation for intricate hero mechanics. The proposed solution introduces a hierarchical framework, multi‑modal feature representation (image‑like, vector, temporal), and multi‑task deep learning models.

The technical solution for Honor Kings AI consists of two main modules: Game Analysis (hero composition, equipment strategy, etc.) and Strategy Execution (deciding next objectives, movement, and combat actions). A global‑view component predicts future hot spots, while a micro‑control component handles combat execution. Both modules use multimodal features and deep multi‑task networks.

In summary, with advances in reinforcement and deep learning, game AI research has shifted from rule‑based to learning‑based approaches. Providing suitable simulation environments is essential for developers to iterate and debug AI systems. MOBA game AI remains challenging but offers abundant opportunities.