Automatic Math Expression Grading with Python, CNN and Image Processing

This article describes a complete workflow for creating an automatic grading system for handwritten arithmetic problems using Python.

Data Generation

Custom fonts are copied into a fonts directory, and a script generates 24×24 pixel images for digits 0‑9 and symbols =, +, -, ×, ÷ by drawing characters with PIL.ImageDraw and applying random rotations.

Model Construction

A small CNN is defined with TensorFlow/Keras: a rescaling layer, two Conv2D‑MaxPooling blocks, a flatten layer, and two dense layers (128 units and 15 output classes). The model is compiled with Adam optimizer and sparse categorical cross‑entropy loss.

Training

The generated images are loaded with tf.keras.preprocessing.image_dataset_from_directory , normalized, shuffled, and cached. The model is trained for 10 epochs, achieving near‑100% accuracy, and the weights are saved to checkpoint/char_checkpoint .

Prediction

Two sample images are loaded with OpenCV, passed through the trained model, and the class with the highest probability is selected, yielding predictions such as ['6', '8'].

Image Segmentation

Projection techniques (horizontal and vertical) are used to locate rows and blocks in a larger image. Functions img_y_shadow and img_x_shadow compute pixel counts per line, and img2rows and row2blocks derive bounding boxes, which are then used to crop the original image.

Recognition of Segments

Each cropped block is fed to the CNN, producing a list of recognized characters for each arithmetic expression.

Calculation and Feedback

The recognized characters are joined into a string and evaluated with Python's eval after replacing "×" with "*" and "÷" with "/". The result is compared to the provided answer; a check mark (green), cross (red), or placeholder (gray) is drawn on the original image using OpenCV and PIL.

Result

The final image shows each expression annotated with the correctness symbol, completing the automatic grading pipeline.