Optimizing NeRF for Real-Time Mobile 3D Rendering in Alibaba's Object Drawer

The article introduces the fourth part of a technical series by Alibaba's Taobao engineers, focusing on the author’s five‑year experience in the Meta team and the challenges of bringing neural radiance fields (NeRF) to industrial scale.

Initially, NeRF offered high‑quality 3D reconstruction but suffered from extremely slow inference (over 1 minute per 1080p frame on a V100 GPU), limiting its use to static rendering. The goal was to achieve real‑time mobile rendering, requiring a speedup of more than 10,000×.

Early attempts using generic acceleration techniques (pruning, 8‑bit quantization, distillation) yielded only ~200× speedup. The author then studied 3D representation and rendering fundamentals, adopting ideas from Fast‑NeRF and PlenOctree, and proposed an Octree + Tiny‑MLP structure combined with SNeRG optimizations, reaching ~6 FPS on a standard Android device (three orders of magnitude improvement).

To further reduce storage and memory (original voxel grids >1.5 GB), a high‑frequency detection algorithm was introduced: low‑frequency regions use fewer voxels, high‑frequency regions use more. Coupled with the HrSRG method (layered voxel representation + perceptual loss + GAN), the voxel count dropped from 60 M to 2 M points, shrinking model size to ~5 MB while preserving texture detail.

Experimental results show clear text and fine details are retained, and the optimized pipeline meets the business requirement of real‑time 3D rendering on low‑end phones. Future work aims to compress models below 1 MB and push frame rates above 50 FPS.

The article also lists related papers (NeRF, PlenOctree, SNeRG, O2U‑Net) and provides links to the team’s other technical showcases.