MIT’s Wave‑Former Reconstructs Fully Occluded Objects with 85% Precision, Boosting Recall to 72%

Problem of Reconstructing Fully Occluded Objects

In computer‑vision and intelligent perception, reconstructing objects that are entirely hidden behind obstacles remains a major challenge. Conventional optical sensors such as cameras or LiDAR cannot see through common occluders like cardboard or fabric, leading to unobservable objects in logistics, manufacturing, or AR scenarios.

Why mmWave and Its Challenges

Millimeter‑wave (mmWave) signals can penetrate many everyday materials and are safe for humans, making them attractive for industrial, logistics, robotics, and AR applications. However, mmWave suffers from strong specular reflections, high noise, and low spatial resolution, which makes direct full‑3D reconstruction difficult.

Wave‑Former: Physics‑Aware Generative AI Solution

MIT researchers propose Wave‑Former, a novel method that embeds the physical characteristics of mmWave into the learning process, bridging wireless perception and modern shape‑completion techniques. The approach consists of two pipelines:

Physics‑aware training pipeline that incorporates three key components:

Mirror‑reflection perception bias replaces the diffuse‑reflection bias of vision‑based models with a specular model matching mmWave physics.

Reflection‑dependent visibility models the anisotropic nature of mmWave returns, using angle‑and‑material‑based attenuation instead of isotropic coverage assumptions.

Joint denoising and completion introduces simulated mmWave noise during training and redesigns the loss so the network directly outputs a complete shape without naïve point‑cloud stitching.

Real‑world inference pipeline with three stages:

Stage 1 – mmWave surface candidate generation: raw mmWave measurements are converted into a set of partial surface patches using recent mmWave imaging techniques.

Stage 2 – Physics‑aware shape completion: each candidate surface is processed by the trained model to produce a physically consistent full reconstruction.

Stage 3 – Entropy‑aware surface selection: local entropy measures the continuity and planarity of point clouds; the candidate with the lowest entropy is chosen as the final high‑fidelity reconstruction.

Training Data and Datasets

Wave‑Former is trained entirely on synthetic data generated from three public 3‑D object datasets:

OmniObject3D – diverse everyday objects (furniture, tools, toys).

Toys4K‑3D – focuses on toys and small items, enriching shape and material diversity.

Objaverse Thingiverse subset – open‑source 3‑D models used to synthesize training samples.

These datasets provide over 25,000 point clouds, enabling rich supervision. For real‑world evaluation, the MITO dataset (61 YCB objects covering kitchen items, tools, food, toys, with materials such as wood, metal, cardboard, plastic) supplies both visible and fully occluded mmWave measurements.

Quantitative and Qualitative Results

Wave‑Former is compared against four state‑of‑the‑art mmWave reconstruction baselines (Backprojection, mmNorm, RMap, RMap‑fine‑tuned). The key metrics are Chamfer Distance (CD), F‑Score, precision, and recall.

Wave‑Former achieves a recall of 72% (up from 54% for the best baseline) while maintaining 85% precision. Its Chamfer Distance drops to 0.069, compared with the best baseline value of 0.18. Qualitative visualizations show Wave‑Former reliably reconstructs complex geometries such as drills and clamps, whereas baselines suffer from low coverage, high noise, or complete failure to resolve shape.

Comparison with Vision‑Based Shape Completion

When evaluated against four leading vision‑based shape‑completion models, Wave‑Former still outperforms them on all metrics, raising recall from 60% to 72% and achieving the highest precision of 85%, demonstrating the benefit of incorporating physical mmWave characteristics.

Ablation Study

The authors analyze the contribution of each design component. Removing the mirror‑reflection bias and reflection‑dependent visibility (Model A) increases average CD by 52% and the 75th‑percentile CD by 67%. Further removing the joint reconstruction and completion module (Model B) adds another 10% CD increase, and eliminating the entropy‑aware surface selection (Model C) raises the 75th‑percentile CD by an additional 19%.

These results clearly illustrate the importance of each component to overall performance.

Technical Extension: From Object Reconstruction to Space Reconstruction

A companion MIT study, RISE (Single Static Radar‑based Indoor Scene Understanding), extends the idea from reconstructing single hidden objects to reconstructing entire indoor spaces using multipath reflections generated by human movement. By feeding low‑quality, sparse mmWave reconstructions into a generative‑AI model, the system learns statistical patterns of multipath reflections and infers full room geometry.

Experiments show RISE reduces Chamfer distance by 60% (to 16 cm) and achieves the first mmWave‑based object detection with IoU = 58%, establishing a new foundation for privacy‑preserving indoor scene understanding.

Implications

Both Wave‑Former and RISE demonstrate a shift: AI is no longer merely enhancing sensor accuracy but is compensating for missing information. By embedding physical priors and leveraging generative models, these systems can infer complete 3‑D structures from highly incomplete, noisy mmWave data, opening new possibilities for robotics, smart homes, and AR.