SpaRV: Sparse Radiance Voxels for Scalable Geometrically Consistent LiDAR-Visual Mapping and Rendering

SpaRV builds a sparse radiance voxel map from LiDAR and cameras. The map stays geometrically consistent away from the captured trajectory. SpaRV also optimizes the camera poses and focal lengths against the map, and reconstructs a university campus captured in 63,582 images on one 24 GB GPU.

Abstract

Robotic simulation, navigation, and inspection need photorealistic and geometrically accurate maps. Radiance fields achieve photorealism, but building them from robot data raises three challenges. A robot's trajectory serves its task, not reconstruction, so images from few viewpoints leave the geometry under-constrained. Residual errors in camera poses and intrinsics blur the map. The map's GPU memory grows with scene size, which limits its level of detail in large scenes. We present SpaRV, a Sparse Radiance Voxel map rendered by differentiable ray marching, which lets irregular LiDAR scans supervise it alongside images. LiDAR depth and occupancy anchor the under-constrained geometry, and normal and smoothness regularizers keep it smooth between LiDAR points. To correct camera errors, SpaRV alternates map training with photometric optimization of the camera poses and focal lengths. To bound GPU memory in training and rendering, SpaRV holds one submap at a time at full resolution and the rest at coarser octree levels. Experiments show that SpaRV keeps the geometry consistent away from the captured trajectory and sharpens textures by optimizing the camera parameters. On one 24 GB GPU, SpaRV reconstructs a university campus captured in 63,582 images and simulates other cameras and LiDARs. We will open-source our implementation.

Video

Method

1Ray-marching rendering of sparse radiance voxels

SpaRV stores the scene as sparse voxels in an octree, allocated only around surfaces and refined where more detail is needed.

A differentiable ray marcher traverses the voxels along each ray independently. Camera pixels, LiDAR rays, and rays of non-pinhole cameras therefore supervise the map through one renderer. The rasterizer of SVRaster, in contrast, batches a pinhole pixel lattice and does not directly apply to LiDAR rays or non-pinhole cameras.

2Geometry-grounded map training

Color supervision fits every camera ray but leaves the geometry under-constrained, since different density profiles can render the same pixel color. LiDAR depth supervision anchors the field at the measured range, but only along measured rays. Between these anchors, normal and smoothness regularizers keep the surface locally smooth.

A LiDAR scan is sparse, and its field of view does not coincide with the camera's. Floaters off the measured rays therefore get no depth supervision. The point cloud still marks which voxels contain a measured surface. Occupied voxel guidance turns that occupancy into a termination target for camera rays that hit an occupied voxel.

Color Depth Normal from depth

Without guidanceWith guidance

The same view on an Oxford Spires scene, from maps trained without and with occupied voxel guidance. The color images are nearly the same, but without the guidance the depth and normal maps show a band of floaters over the lawn.

3Map-driven camera optimization

The cameras of robot data are metrically plausible but carry residual errors in every frame, and these errors blur the textures of the map. SpaRV alternates map training with photometric optimization of the pose and focal length of each camera against the current map. On the six FAST-LIVO2 scenes, camera optimization raises the training-view PSNR of SpaRV from 26.55 dB with the odometry cameras to 29.87 dB.

Drive Sculpture CBD Culture

Odometry camerasOptimized cameras

Show optimized cameras vs. ground truth

Optimized camerasGround truth

SpaRV on a FAST-LIVO2 training view, trained with the odometry cameras (left) and with the cameras it optimized (right).

Odometry cameras: every method is trained on the FAST-LIVO2 odometry poses, and SpaRV (cam. opt.) also runs camera optimization. Optimized cameras: every method is trained on the cameras that SpaRV (cam. opt.) optimized.

4Large-scale reconstruction under bounded GPU memory

The GPU memory of a map grows with the extent of the scene. A single view, however, needs fine detail only near the camera, since distant regions project to few pixels. The octree of SpaRV keeps every coarser level that training passes through, and a coarser voxel needs no further optimization to be rendered as a level of detail. A granularity-driven cut selects voxels by the number of pixels they span on the image plane. It holds the submap in focus at the finest available level and the rest of the scene at coarser levels. The same cut rule serves training and rendering.

Levels of detail of one octree, seen from a single viewpoint. Top: increasingly fine levels. Bottom: cuts at an increasing granularity target τ, in pixels.

Results

We compare SpaRV with InstantNGP, 3DGS, SVRaster, LetsGo, and GS-SDF on synthetic indoor scenes and two outdoor LiDAR-visual datasets, with H3DGS on two large scenes we collected, and with EgoNeRF and OmniGS on an omnidirectional benchmark. In the tables, bold is the best result and underlined the second best.

Rendering away from the captured trajectory

A robot plans, simulates, and navigates beyond the views it captured, so the map is expected to stay geometrically consistent at those viewpoints. On FAST-LIVO2 Street, the methods are hard to tell apart at an interpolated view and separate once the camera moves off the captured trajectory. The image-only methods (InstantNGP, 3DGS, SVRaster) degrade. The Gaussian-based LetsGo and GS-SDF show popping on the billboard, where SpaRV stays consistent.

3DGSSpaRV (Ours)

Compare with: 3DGS InstantNGP SVRaster LetsGo GS-SDF

FAST-LIVO2 Street at an extrapolated view away from the captured trajectory. Pick a baseline and drag the divider.

Replica

Replica is a synthetic indoor dataset with ground-truth camera poses, which remove camera error from the comparison. We evaluate 8 scenes at interpolated viewpoints along the trajectory and at extrapolated viewpoints sampled uniformly in each scene. Pixels that no training view sees are excluded from the extrapolated evaluation.

Extrapolated view Interpolated view

3DGSSpaRV (Ours)

Compare with: 3DGS InstantNGP SVRaster LetsGo GS-SDF

Replica Room-0. Every method is close to the reference at the interpolated training view; at the extrapolated view of the same maps, the methods separate.

FAST-LIVO2

FAST-LIVO2 is a real-world LiDAR-visual dataset captured with a solid-state Livox AVIA LiDAR. Its non-repetitive scan pattern makes the range measurements of each frame sparse and irregular. We use six outdoor scenes with three trajectory types: forward-facing (Landmark, CBD, Street), object-centric (Sculpture), and free-view (Culture, Drive). The initial cameras come from the FAST-LIVO2 odometry.

Fly-throughs along the test-view trajectory, with every method trained on the optimized cameras. Pick a scene and a baseline, and drag the divider.

On a test split that holds out one image in every eight, SpaRV has the lowest LPIPS on all six scenes and the highest SSIM on five of them. Its average PSNR is 0.13 dB below GS-SDF.

Oxford Spires

Oxford Spires has 12 outdoor sequences around Oxford landmarks, captured with a 64-channel Hesai QT64 LiDAR and three synchronized global-shutter cameras. Its ground-truth trajectories are registered against a millimeter-accurate laser scan, so this benchmark compares the methods with accurate cameras. SpaRV leads on all three averaged metrics, and its LPIPS is the lowest on all 12 sequences.

Keble-05 Obs.-01 Bodleian-02

3DGSSpaRV (Ours)

Compare with: 3DGS InstantNGP SVRaster LetsGo GS-SDF

Large scenes: Garden and Campus

We captured Garden and Campus with a handheld scanner that carries a solid-state Livox MID360 LiDAR and three fisheye cameras. Garden is an outdoor district densely covered with complex objects: 44,550 images and 146.7 M LiDAR points along a 2,239 m trajectory. Campus spans an entire university: 63,582 images and 151.6 M LiDAR points along a 3,534 m trajectory. We compare SpaRV at two stages with H3DGS. SpaRV (coarse) is the output of coarse training, and SpaRV (deploy) adds submap training and renders through a cut around the camera.

SpaRV (coarse), H3DGS, and SpaRV (deploy) along the whole fly-through of each scene. Campus plays at 2–3× speed.

Submap training improves SpaRV (coarse) on every metric, most on Campus (+2.5 dB PSNR). On Campus, SpaRV (deploy) renders at 68.8 FPS with a peak of 15.6 GiB of GPU memory, against 29.6 FPS and 22.5 GiB for H3DGS.

Omnidirectional cameras: EgoNeRF

SpaRV renders equirectangular cameras with the same ray marcher, and uses the renderer and training schedule of the pinhole experiments unchanged. OmniGS, in contrast, specializes its rasterizer for the equirectangular camera. SpaRV outperforms OmniGS on all three metrics on both splits. The PSNR margin is small (0.34 dB on OmniBlender, 0.15 dB on Ricoh360), while LPIPS drops by 26% and 11%.

Applications

Sensor simulation

The ray marcher accepts arbitrary rays, so one trained map can render other cameras and LiDARs. From the Garden map, SpaRV renders three camera models (pinhole, fisheye, and omnidirectional) and three LiDAR scan patterns (Livox AVIA, Livox MID360, and an OS128-style ring pattern).

Mesh extraction

Meshes extracted from the six FAST-LIVO2 maps by marching cubes.

BibTeX

@misc{liu2026sparv,
  title   = {SpaRV: Sparse Radiance Voxels for Scalable Geometrically Consistent LiDAR-Visual Mapping and Rendering},
  author  = {Liu, Jianheng and Wan, Yunfei and Cheng, Fangming and Hong, Xiangyu and Zou, Zuhao and
             Li, Haotian and Yin, Longji and Zheng, Chunran and Lin, Jiarong and Zhang, Fu},
  note    = {Under review},
  year    = {2026}
}