Build a DIY AI Bot for Honor of Kings Using Transformers and scrcpy

Overview

Jack introduces a "civilian version" of an AI that can play the popular mobile game Honor of Kings . The solution combines scrcpy for screen mirroring and control, minitouch for touch‑event injection, and a Transformer model that converts game screenshots into actionable commands.

Principle

To control the game from a PC you need two tools:

scrcpy : projects the Android screen onto the computer, allowing mouse, keyboard, clipboard, file drag‑and‑drop, and APK installation without rooting the device.

minitouch : provides a socket interface for multi‑touch events and gestures, also without requiring root.

The workflow is: capture the game screen with scrcpy, feed the image to a Transformer that generates a textual description of the required action, translate that description into touch commands, and send them to the device via minitouch.

Why a Transformer?

Transformers excel at mapping visual inputs to language outputs. By training a model on paired screenshots and action descriptions (e.g., "move forward", "attack", "use skill"), the AI can infer the correct sequence of touches for any game state.

AI‑generated image example

Implementation Steps

1. Prepare a GPU‑enabled machine and install required Python libraries such as pyminitouch and torch. 2. Install scrcpy (many tutorials are available online) and place the executable in the same directory as the code. 3. Download the pre‑trained model files (provided by the author) and extract them alongside the scripts. 4. Adjust the resolution settings in the code if your phone differs from the reference 1080×2160 screen. 5. Run 启动和结束进程.py to start scrcpy. 6. Edit 训练数据截取_A.py to match your device identifiers, then launch it while playing a 5v5 AI match to collect training data. 7. Train the Transformer model on the captured data, then use it to generate real‑time action commands during gameplay.

Final Thoughts

The project code is somewhat messy and contains Chinese variable names, and the AI’s performance is still limited to basic bot play. Future work will focus on cleaning the codebase, improving the model, and extending support to more heroes.

Project repository: https://github.com/FengQuanLi/ResnetGPT