How GaussianPile Enables 3DGS to Reconstruct Internal Structures from Slice‑Based Volumetric Images

Background

3D Gaussian Splatting (3DGS) has become a powerful representation for rendering natural‑scene surfaces, but its standard formulation assumes an all‑in‑focus camera and does not account for the physical constraints of slice‑based volumetric imaging such as ultrasound, light‑sheet microscopy, or MRI.

Why a New Representation Is Needed

Slice‑based biomedical data present three challenges:

Massive data volume: high‑resolution voxel grids grow prohibitively expensive as resolution increases.

Finite slice thickness and depth of field: each acquired slice integrates signal over a non‑negligible thickness, especially in ultrasound and light‑sheet microscopy.

Interactive efficiency: many implicit neural representations compress well but are too slow for real‑time browsing or robot‑assisted surgery.

GaussianPile is designed to address these issues by preserving internal structure while remaining compact and fast.

Core Idea of GaussianPile

The framework explicitly writes the “finite thickness” of slices into the Gaussian rendering process. Standard 3DGS assumes a thin‑slice, all‑in‑focus model; applying it directly to volumetric data can fit 2D slices yet produce unreliable 3D internal structures. GaussianPile introduces a focus‑aware physical model that treats the point‑spread function along the slice axis as a sensitivity map and modulates each Gaussian’s contribution based on its distance to the current slice.

Focus‑Aware Rendering Pipeline

The pipeline consists of three steps:

Scan : sample virtual slices at different depths from the 3D Gaussian representation.

Focus : re‑parameterize Gaussians axially according to slice thickness and system depth of field, attenuating out‑of‑focus contributions.

Pile : project the focus‑modulated Gaussians onto 2D slices using additive rasterization, which accumulates intensity rather than alpha blending to match volumetric signal integration.

From Voxel Grid to Sparse Gaussian Representation

GaussianPile does more than compress; it converts slice sequences into a continuous, sparse Gaussian intermediate. Each Gaussian stores position, scale, orientation, and intensity. Because Gaussians are continuous functions, they avoid per‑voxel storage. The framework also removes spherical‑harmonic coefficients used for surface color in standard 3DGS, as medical slice intensity is view‑independent.

During compression, Gaussian parameters are quantized and entropy‑coded, exploiting spatial correlation. Experiments report a stable ~16× compression versus voxel grids, with peak compression ratios of 20–26× on certain datasets, while preserving high‑quality 2D slice reconstruction and consistent 3D structure.

Experimental Evaluation

GaussianPile was evaluated on automatic breast ultrasound (ABUS), light‑sheet microscopy (LSM), and multi‑cell microscopy datasets. It was compared against HEVC, INR/NeRF‑style compression methods, and the original 3DGS pipeline.

Results show higher 2D/3D PSNR and SSIM across all datasets. Compared with HEVC, GaussianPile retains finer structures; versus INR methods, it offers better high‑frequency detail and faster optimization; versus vanilla 3DGS, it eliminates 3D inconsistency and floating‑artifact issues.

Training converges in about 8 minutes on average, with some cases reaching satisfactory reconstructions in ~3 minutes, delivering roughly an 11× speedup over INR/NeRF approaches while maintaining real‑time rendering capability.

Scalability tests on large‑scale electron microscopy data demonstrate that the method remains effective at higher resolutions, suggesting applicability to both small‑sample and large‑scale scientific imaging.

Future Directions

In robot‑assisted surgery, rapid 3D reconstruction of tissue is essential for planning and execution. GaussianPile offers a compromise: it transforms raw slice data into a compact, continuous Gaussian representation without altering the underlying imaging modality, supporting fast browsing, compression, voxel‑based evaluation, 3D segmentation, and potential deformable modeling.

Conclusion

GaussianPile shows that 3DGS can be extended from natural‑scene surface rendering to slice‑based volumetric imaging by incorporating finite slice thickness and depth‑of‑field effects into the forward projection. The method achieves a balanced trade‑off among reconstruction quality, training speed, and compression ratio, opening new pathways for medical image compression, scientific data visualization, and real‑time 3D perception in surgical robotics.