CVPR 2026: Oxygen XR Team Unveils DirectFisheye‑GS for Native Fisheye Gaussian Splatting

Traditional Gaussian splatting relies on pinhole camera inputs, which limits field of view and requires many images to fully cover a scene. Existing fisheye‑based solutions typically perform undistortion preprocessing, leading to information loss, blurred details, and edge artifacts.

DirectFisheye‑GS, proposed by the Oxygen XR research team together with Tsinghua University, integrates a native fisheye projection model into the 3D Gaussian splatting pipeline. By embedding a differentiable fisheye projection and a cross‑view joint optimization strategy, the method directly processes fisheye images without undistortion, solving Gaussian shape distortion caused by single‑view random optimization and ensuring geometric and photometric consistency across views.

The pipeline starts with an initialization stage that creates a 3D Gaussian ellipsoid for each sparse input point, defined by seven core parameters (center, 3×3 covariance matrix, and RGB color). A differentiable tile rasterizer renders these Gaussians onto the screen, producing 2D images while allowing loss computation against ground‑truth pixels. Optimization updates all Gaussian parameters via back‑propagation. An adaptive density control mechanism evaluates the importance of each Gaussian during training; important Gaussians are split to increase local detail density, while redundant ones are pruned, keeping the model efficient yet detailed.

Experiments were conducted on a single NVIDIA A100 80 GB GPU using three public datasets—FisheyeNeRF, ScanNet++, and Den‑SOFT—covering small objects, medium‑scale indoor scenes, and large‑scale outdoor environments. Baselines included the original 3DGS, Fisheye‑GS, 3DGUT, and Self‑Cali‑GS. Evaluation metrics (PSNR, SSIM, LPIPS) show that DirectFisheye‑GS achieves or surpasses the current state‑of‑the‑art on all training and testing viewpoints, with especially large PSNR gains in regions where fisheye distortion exceeds 60°. Qualitative results demonstrate the removal of edge mosaics, texture blur, and floating Gaussians, delivering superior detail preservation and cross‑view lighting consistency.

Beyond the technical contribution, the authors discuss broader applications in e‑commerce and content creation, where 3D static reconstruction and emerging 4D volumetric video can enhance immersive shopping and interactive media. Current limitations include the need for multi‑view capture with complex lighting and reconstruction times ranging from tens of minutes to hours. Future work aims to leverage advancing edge‑AI compute, algorithmic efficiency, and hardware integration to achieve one‑click capture and near‑real‑time reconstruction.