Applying OCR to Game Skin Recognition: Filtering Owned Skins and Tolerant Text Matching

OCR (optical character recognition) converts text in images into machine‑readable strings and is widely used in various business scenarios, such as recognizing address information in delivery apps.

In the context of a game marketplace, users upload images of game skins; manually filling in dozens of skin parameters is cumbersome, so OCR is employed to extract these parameters automatically.

The main challenges are (1) recognizing only the skins that the user actually owns and (2) tolerating OCR misrecognitions caused by complex characters.

To separate skin regions from the background, the image is analyzed for color diversity: background areas have uniform colors, while skin regions contain richer, varied colors. By scanning rows and columns and counting distinct colors, sharp changes indicate region boundaries, allowing the image to be divided into rectangular skin blocks.

After region segmentation, owned skins are identified by evaluating the brightness and saturation of pixels within each block. Owned skins tend to have higher‑intensity, high‑saturation pixels, whereas unowned skins appear darker. The following Java method computes a score based on these thresholds:

private double colorScore(BufferedImage img, Rectangle rect, double brightnessThreshold, double saturationThreshold) {
    int total = 0;
    Set<Integer> set = new HashSet<>();
    for (int i = (int) rect.getX(); i < (int) (rect.getX() + rect.getWidth()); i++) {
        for (int j = (int) (rect.getY() + rect.getHeight() / 4); j < (int) (rect.getY() + rect.getHeight() * 3 / 4); j++) {
            int color = img.getRGB(i, j);
            int r = (color >> 16) & 0xff;
            int g = (color >> 8) & 0xff;
            int b = color & 0xff;
            double max = Math.max(r, Math.max(g, b));
            double min = Math.min(r, Math.min(g, b));
            double saturation = 1 - min / max;
            double brightness = 0.3 * r + 0.6 * g + 0.1 * b;
            if (brightness > brightnessThreshold && saturation > saturationThreshold) {
                set.add(color);
            }
            total++;
        }
    }
    if (total == 0) return 0;
    return set.size() / (double) total;
}

For tolerant text matching, a Rabin‑Karp based sliding‑window hash approach is used. The algorithm splits each OCR result into overlapping substrings, hashes them, and compares the hash sets to determine similarity.

public static void main(String[] args) {
    List<String> list1 = Arrays.asList("李信", "信一", "一念", "念神", "神魔");
    List<String> list2 = Arrays.asList("李信", "信一", "一念", "念神", "神度");
    Set<String> baseSet = new HashSet<>(list2);
    float count = 0;
    for (String word : list1) {
        if (baseSet.contains(word)) count += 1;
    }
    System.out.println(count / list1.size()); // prints 0.8
}

The hash generation for a sliding window is implemented as:

public static List<Long> hashCode(String str, int windowLen, int moveLen) {
    List<Long> hashList = new ArrayList<>();
    int temp = 1;
    for (int i = 0; i < windowLen; i++) {
        temp = (temp * BASE_NUM) % MOD_NUM;
    }
    long curHash = 0;
    for (int i = 0; i < str.length(); i++) {
        curHash = (curHash * BASE_NUM + str.codePointAt(i)) % MOD_NUM;
        if (i > windowLen - 1) {
            curHash -= (str.codePointAt(i - windowLen)) * temp % MOD_NUM;
        }
        if (curHash < 0) curHash += MOD_NUM;
        if ((i + 1) % moveLen == 0 || (i + 1) == str.length()) {
            hashList.add(curHash);
        }
    }
    return hashList;
}

During system startup, the parameter library is pre‑processed to generate hash collections for each known skin name. When OCR produces a result, its hash collection is compared against the stored ones; a similarity above a configured threshold confirms a match.

After implementing both owned‑skin filtering and tolerant matching, the solution was deployed for two games (King of Glory and Peace Elite), automatically enriching thousands of product entries daily, with plans to extend to additional titles.