From Compression to Memory MonkeyOCRv2 Reconstruction Preserves Document Evidence

Understanding how much a document visual encoder contributes to downstream performance is essential. In a tightly controlled experiment where the language model (Qwen3‑1.7B), training data, optimizer settings, and decoding pipeline are frozen, only the visual encoder varies. MonkeyOCRv2‑B achieves 57.2 points on eight document‑understanding benchmarks, a 13.2‑point lead over the strongest baseline OpenVision‑B, proving that the visual front‑end alone can determine a two‑digit performance gap.

Reconstruction as Visual Memory

MonkeyOCRv2 is trained jointly on image‑to‑text generation and pixel‑level reconstruction. Generation cares only about the final text, ignoring font, spacing, or layout variations. Reconstruction forces the same visual tokens to retain enough information to recreate the original page, exposing any loss of character strokes, formula symbols, or layout details.

Evidence Chain – First Ring: Freezing the Backend

By fixing the language model and downstream components, the experiment isolates the visual encoder as the sole variable. The resulting 13.2‑point improvement confirms that the visual encoder is not a replaceable front‑end but a decisive factor for document AI.

Second Ring: Removing Language Shortcuts

To test reliance on language priors, the authors create two versions of the same text: one preserving normal semantics, the other with characters randomly shuffled, destroying contextual cues. At the lowest resolution (448 px long side), a model without reconstruction attains 55.4 % accuracy on shuffled text, while adding reconstruction raises it to 72.1 % (+16.7 pts). The gap between normal and shuffled text shrinks from 29.3 pts to 15.3 pts, showing that reconstruction supplies useful visual evidence when language help is absent.

Third Ring: CHAOS‑Bench – Visual Evidence vs. Language Priors

CHAOS‑Bench deliberately alters a few characters on the page to create a visual‑language conflict. Models that rely heavily on language may “correct” the error, whereas a faithful visual model should output the exact characters. MonkeyOCRv2‑B‑Parsing achieves a page‑level recall of 17.9, surpassing HunyuanOCR‑1.5 (14.2), DeepSeek‑OCR 2 (6.3), MinerU2.5‑Pro (6.3) and PaddleOCR‑VL‑1.6 (6.0). An ablation shows that adding reconstruction to MonkeyOCRv2‑S improves recall from 12.1 to 14.7.

Cross‑Task Transfer – Seven Document Tasks

MonkeyOCRv2 serves as a universal visual backbone for text recognition, formula recognition, text detection, document tampering detection, overlapping‑text segmentation, parsing, and understanding. Across these tasks, it consistently adds performance gains: CRNN +2.3 pts, PARSeq +0.4 pts (surpassing SVTRv2), UniMERNet‑S (1.1 B parameters) outperforms the larger UniMERNet‑B (3.25 B), and improvements of 3.3 pts (detection), 7.5 pts (tampering), and 5.3 pts (segmentation) are reported.

System‑Level Leaderboard

On the official MDPBench leaderboard, MonkeyOCRv2‑B‑Parsing (0.7 B total parameters) scores 83.3 overall, with 81.7 on photographed documents and 82.1 on non‑Latin scripts, making it the top open‑source model among evaluated systems.

Limitations and Future Directions

While reconstruction proves beneficial, the evidence chain is not fully closed: it remains unclear whether pixel‑level reconstruction is the optimal memory mechanism, and MSE loss may retain irrelevant low‑level details. Future work could compare alternative reconstruction targets, analyze token‑compression trade‑offs, probe character‑level information with linear probes, and evaluate on low‑resource, handwritten, or historical documents.

MonkeyDoc v2 – A Public Large‑Scale Document Dataset

To enable fair, reproducible research, the authors release MonkeyDoc v2, containing 1.13 B document images covering 17 languages, including 800 M page‑level images and 1.05 B fine‑grained elements (text, tables, formulas). This dataset supports page‑level parsing, layout analysis, and end‑to‑end recognition, providing a common starting point for investigating questions such as the contribution of generation vs. reconstruction, the optimal mix of page‑level and cropped data, and the performance of smaller encoders with better pre‑training objectives.

Conclusion – From Compression to Memory

Document AI should move beyond merely compressing a page into fewer visual tokens; it must ensure that the compressed representation can faithfully reconstruct the critical visual evidence. MonkeyOCRv2’s reconstruction‑based visual memory demonstrates that preserving fine‑grained page details improves both recognition and higher‑level understanding, pointing toward a future where “visual memory” is a core design principle.