How APEIRIA Breaks the Black‑Box Barrier of 3D MLLMs (ICML 2026)

The authors identify a core dilemma in 3D spatial reasoning: 3D multi‑modal large language models (3D MLLMs) can interpret open‑world language but operate as black‑box end‑to‑end mappings, whereas neuro‑symbolic 3D methods offer transparent, step‑by‑step verification but rely on closed vocabularies and dense supervision.

To bridge this gap, they propose APEIRIA, a neuro‑symbolic 3D MLLM that distills the systematic spatial reasoning pattern of symbolic programs into a large model. The key insight is that the valuable transferable component is the *spatial reasoning mode*—how queries are decomposed, objects are located, relations are verified, and intermediate states are composed into final answers.

Three‑stage curriculum learning injects this reasoning mode:

Stage 1 – 3D Perception Alignment: The model learns to “see” the 3D world by aligning object recognition, attribute understanding, and pose prediction with textual space, establishing basic scene comprehension.

Stage 2 – Symbolic Reasoning Injection: Verified execution traces from neuro‑symbolic programs are serialized into natural‑language chain‑of‑thought (CoT) narratives, providing precise step‑level supervision (object IDs, coordinates, sizes, relational judgments).

Stage 3 – CoT‑RL: Because full step‑wise supervision is unavailable in real data, reinforcement learning uses only final 3D reasoning outcomes and format constraints as rewards, extending the learned reasoning pattern to open‑vocabulary and nested instructions.

The distilled reasoning retains explicit planning and execution separation, allowing plug‑and‑play upgrades of perception or planning modules without retraining the language model.

Experimental results (Table 1) show APEIRIA surpasses or matches state‑of‑the‑art 3D MLLM baselines on ScanRefer and Multi3DRefer. Adding modular perception enhancements further improves performance.

Zero‑shot open‑vocabulary tests (Table 2) demonstrate that training only on synthetic commands enables the model to generalize to natural‑language instructions, confirming that the learned reasoning pattern, not a closed concept set, transfers.

Ablation studies (Table 3) reveal that removing the CoT‑RL stage or skipping symbolic reasoning injection causes significant drops, highlighting the necessity of each curriculum component for stable 3D reasoning.

Modular upgrades (Table 4) replace the perception module with a stronger SegDINO3D model, yielding consistent gains across benchmarks, proving that the bottleneck lies in visual perception rather than planning.

Qualitative analysis (Figure 3) shows emergent logical operations such as intersection and union when handling multi‑condition queries like “this beige chair is next to the coat rack and to the left of the table and lamp,” indicating the model internalizes spatial logic beyond rote program templates.

Overall, APEIRIA provides a concrete pathway to combine the open‑semantic power of 3D MLLMs with the interpretability and modularity of neuro‑symbolic reasoning, advancing toward explainable and upgradable embodied AI agents.