Using PyAutoGUI for Cross‑Platform GUI Automation in Python

Follow + Star, Learn New Python Skills Daily

Many users have tried tools like AutoHotkey or similar key‑macro software, but with Python you can achieve the same automation using the PyAutoGUI library, which works on Windows, Linux, and macOS.

Installation and Usage

Install PyAutoGUI via pip:

<code>pip install pyautogui</code>

The library uses a screen coordinate system where the top‑left corner is (0,0); the x‑axis increases to the right and the y‑axis increases downward. For a 1920×1080 screen, the bottom‑right corner is (1919,1079).

Function Overview

PyAutoGUI provides simple functions for mouse actions (click, double‑click, drag, scroll), keyboard actions (press, hotkey combinations), dialog boxes (alert, confirm, prompt, password), and screenshot capabilities (capture the screen or locate images on the screen).

Example: Drawing with PyAutoGUI

<code>import pyautogui

distance = 100
pyautogui.moveTo(400, 300)
while distance &gt; 0:
    pyautogui.drag(distance, 0, duration=0.1)
    distance -= 5
    pyautogui.drag(0, distance, duration=0.1)
    pyautogui.drag(-distance, 0, duration=0.1)
    distance -= 5
    pyautogui.drag(0, -distance, duration=0.1)
</code>

The script moves the mouse and draws concentric squares, demonstrating how simple the API is.

Example: Image‑Based Button Clicking

<code>import pyautogui
import time

time.sleep(3)

# Paths to button images
five = '5.png'
eight = '8.png'
multiply = 'multiply.png'
equals = 'equals.png'

def find_and_click(image):
    x, y = pyautogui.locateCenterOnScreen(image, confidence=0.9)
    pyautogui.click(x, y)

# Perform 5 * 8 =
find_and_click(five)
find_and_click(multiply)
find_and_click(eight)
find_and_click(equals)
</code>

This example requires OpenCV (opencv‑python) for image recognition and demonstrates how to locate and click UI elements even when the match is not perfect.

Full source code and additional samples are available on the author's GitHub repository:

<code>https://github.com/techstay/python-study/tree/master/pyautogui-sample</code>

Using PyAutoGUI, you can automate graphical user interfaces by controlling mouse and keyboard inputs based on screen coordinates or image recognition, offering a powerful alternative to traditional macro tools.