How AI Powers Automatic Homework Grading: Challenges and Solutions

Automatic homework grading is a key AI application in education, enabling personalized teaching and reducing student workload. It consists of six modules: preprocessing, layout analysis, OCR recognition, answer matching, grading strategy, and result display (see Figure 1).

The grading pipeline follows eight steps: capture a photo on a mobile device, upload to the server, obtain a high‑quality image via preprocessing, split the image by question type through layout analysis, extract all information with the recognition module, integrate the data, apply a grading strategy, and finally present the results (Figure 2). Implementing the full pipeline typically requires around ten models, and the output is illustrated in Figure 3.

Questions are grouped into three categories based on grading difficulty. Category 1 includes logically gradeable items such as elementary arithmetic expressions and equations; Category 2 contains text‑rich questions with a unique answer and relatively fixed answer positions (e.g., multiple‑choice, fill‑in‑the‑blank, true/false, application problems); Category 3 comprises graphic‑oriented items like drawing, selection, or linking tasks, whose answers are not easily formalized and thus pose the greatest challenge.

For Category 1, the workflow uses a detection model to locate each question and its text line, an OCR model to read printed and handwritten characters, and then computes the answer directly for comparison. Issues such as illegible handwriting or poor photo quality can degrade performance.

Category 2 requires semantic understanding. Two approaches exist: automatic solving with OCR + NLP, which is still immature, or a large curated answer database combined with text and image search. This pipeline involves many sub‑tasks (image correction, handwriting removal, OCR, semantic similarity, etc.) and suffers from layout variations, non‑unique answers, and mismatched answer ordering.

Category 3 currently lacks effective solutions because answers cannot be easily formalized. Even with a matching question in a database, comparing answers is difficult, and labeling the correct answer is itself a problem. Ongoing advances in computer vision and multimodal models may eventually enable a formal representation of such answers.

In summary, while the overall grading process is similar across question types, the implementation gaps are significant. The team has made substantial progress on the first two categories and is actively researching strategy modeling and solutions for the third category, anticipating breakthroughs in the near future.