Building an Automated Minesweeper Solver with Python and OpenCV

Introduction: The author demonstrates how to break a world record by building an automated Minesweeper solver with Python and OpenCV.

Required environment: Python 3.6+, preferably Anaconda3, and libraries numpy, PIL, opencv‑python, win32gui, win32api, plus any IDE.

Minesweeper software: Use Minesweeper Arbiter as the target game.

Implementation steps are labeled 0x00 (preparation) and 0x01 (idea). The workflow consists of four main stages: window capture, block segmentation, block recognition, and solving algorithm.

Window capture: Using win32gui to locate the game window by class name "TMain" and title "Minesweeper Arbiter ", then obtain its rectangle and crop the board with PIL.

hwnd = win32gui.FindWindow(class_name, title_name)
if hwnd:
    left, top, right, bottom = win32gui.GetWindowRect(hwnd)

left += 15
top += 101
right -= 15
bottom -= 43
rect = (left, top, right, bottom)
img = ImageGrab.grab().crop(rect)

Block segmentation: Each cell is 16×16 pixels. The number of cells in x and y directions is computed, and the board image is sliced into a 2‑D list.

block_width, block_height = 16, 16
blocks_x = int((right - left) / block_width)
blocks_y = int((bottom - top) / block_height)

def crop_block(hole_img, x, y):
    x1, y1 = x * block_width, y * block_height
    x2, y2 = x1 + block_width, y1 + block_height
    return hole_img.crop((x1, y1, x2, y2))

Block recognition: The centre pixel colour of each cell determines its state. The author maps specific BGR values to numbers 1‑8, mines (9), flags (0), unopened (-1), blank (-2), and unknown (-3).

block_color = block[8, 8]
if self.equal(block_color, self.rgb_to_bgr((192, 192, 192))):
    if not self.equal(block[8, 1], self.rgb_to_bgr((255, 255, 255))):
        self.blocks_num[x][y] = -2
        self.is_started = True
    else:
        self.blocks_num[x][y] = -1
# … (other colour checks omitted for brevity)

Solving algorithm: The solver iterates over numbered cells, generates a 3×3 kernel around each cell, counts unopened neighbours, and applies three rules – mark mines when the count matches the number, click safe cells when the remaining count matches, and resort to random clicks when no deterministic move exists.

def generate_kernel(k, k_width, k_height, block_location):
    ls = []
    loc_x, loc_y = block_location[0], block_location[1]
    for now_y in range(k_height):
        for now_x in range(k_width):
            if k[now_y][now_x]:
                rel_x, rel_y = now_x - 1, now_y - 1
                ls.append((loc_y + rel_y, loc_x + rel_x))
    return ls

if unopen_blocks == self.blocks_num[x][y]:
    mark_as_mine(to_visit)

Finally, the program moves the mouse to the selected safe cells using win32api calls encapsulated in mouseOperation.py and performs clicks.

on_screen_location = self.rel_loc_to_real(to_click)
mouseOperation.mouse_move(on_screen_location[0], on_screen_location[1])
mouseOperation.mouse_click()

The article ends with a note that the code works on Windows 10 and invites readers to explore the full source on the linked GitHub repository.