How to Automate Onmyoji Gameplay on Android with Python, ADB, and OCR

Personal Usage Order

Each week start by running the secret dungeon, then daily three regional bosses, use barrier breakthrough tickets when full, and when tickets run out, run soul collection; roughly 80‑100 cycles fill the tickets, then continue with breakthroughs.

Idea

Capture screenshots via ADB, then use OpenCV for image matching and color detection to simulate clicks; ADB operates the emulator in the background.

Target Features

The tool focuses on regional bosses, barrier breakthroughs, and secret dungeons to farm jade; soul collection is used to obtain breakthrough tickets.

Random Delays

Random wait times of 2‑3 seconds are added to avoid detection, slightly slowing overall speed.

Region Boss

Requires prior collection of specific characters (e.g., Gu Hu Niao, Ji Tian Zhen Tian, Shan Tong).

Barrier Breakthrough

Coordinates of the 3×3 grid are stored in a list, shuffled with shuffle, and looped through; only personal barrier breakthroughs are supported and the lineup must be unlocked.

Tencent OCR provides 1,000 free calls per month, sufficient for breakthrough ticket recognition; additional usage can be paid or custom dictionaries can be built.

OCR Setup

Apply for Tencent Cloud OCR secretId and secretKey, then create tencentcloudKeys.py containing the credentials.

secretId = "AKI***********************aYHDtmaOw"
secretKey = "Sg**************************QdZ7X"

Emulator Version

Recommended emulator: LDPlayer (or NetEase MuMu). Resolution: 1024×576. MuMu graphics mode: Compatibility (OpenGL). Works on Windows and macOS.

Running Onmyoji

Find the app's launch command with adb shell dumpsys window | findstr mCurrentFocus, then start the game with:

adb shell am start -n com.netease.onmyoji.netease_simulator/com.netease.onmyoji.Client

Simulate Click

Example to click at coordinates (500, 266):

adb shell input tap 500 266

Simulate Swipe

Swipe from (0,0) to (200,200) over 0.5 seconds:

adb shell input swipe 0 0 200 200 500

Screenshot

Capture the screen and pull it to the PC:

adb shell screencap /data/screen.png
adb pull /data/screen.png .

Note: The command saves the image in the current directory.

Image Matching (Find Image)

Use OpenCV to compare a live screenshot with a template image:

import cv2

def p():
    capture_img = ""
    temp_img = ""
    img1 = cv2.imread(capture_img)
    img2 = cv2.imread(temp_img)
    result = cv2.matchTemplate(img1, img2, cv2.TM_CCOEFF_NORMED)
    if result.max() > 0.9:
        return True

Color Detection (Find Color)

Get the RGB value of a pixel at (x, y) in a screenshot:

import cv2

def p(x, y):
    capture_img = ""
    _img = cv2.imread(capture_img)
    img = cv2.cvtColor(_img, cv2.COLOR_BGR2RGB)
    r, g, b = img[y, x]
    return r, g, b

Crop Image

Crop a region from a screenshot (coordinates are y:x):

import cv2
capture_img = ""
img = cv2.imread(capture_img)
cv2.imwrite("new.png", img[12:30, 705:750])

Dual‑App Switching

After multi‑instance modification, original and cloned apps share the same package name; control them via UserId. Example for NetEase Cloud Game:

adb shell dumpsys window | findstr mCurrentFocus
adb shell ps|findstr com.netease.android.cloudgame

Original app UserId is 0; cloned instances use incremental UserIds (e.g., 10, 11, 12...). Switch UserId and start the app:

# Switch to original app
adb shell am start-user 0
adb shell am start --user 0 com.netease.android.cloudgame/com.netease.android.cloudgame.MainActivity

# Switch to cloned app (UserId 10)
adb shell am start-user 10
adb shell am start --user 10 com.netease.android.cloudgame/com.netease.android.cloudgame.MainActivity