MR-Compare

MR-Compare is a spatially grounded mixed reality framework for comparing heterogeneous 3D reconstructions with the live physical world. Implemented on Meta Quest 3, the system aligns mesh and 3D Gaussian Splatting reconstructions to the headset's world coordinate system and enables cross-media comparison between reconstructed scenes and live video see-through.

The paper evaluates MR-Compare through a real-world benchmark and an exploratory user study with 30 participants, covering practical desktop and mobile reconstruction workflows. The results show centimetre-level registration across tested workflows and strong visual consistency for desktop 3DGS-based reconstructions.

This demo shows the final MR-Compare outcome in an office scene. Heterogeneous 3D reconstructions have already been registered to the physical space, allowing the user to switch between the aligned reconstruction and the live environment, and to inspect the result with the 3D Slider.

Final outcome demo: five registered reconstruction outputs aligned with the physical office, including standard 3DGS, RealityScan dense mesh, Scaniverse 3DGS splat, Polycam mesh, and 3DGS-MCMC. The demo shows switching between reconstructions and visualization with the 3D Slider.

The teaser summarises MR-Compare as a single source-to-interaction pipeline. The system first converts heterogeneous reconstructions and headset observations into comparable point-cloud representations, then uses coarse-to-fine registration to place the reconstruction in the Quest world coordinate system before enabling in-headset visual comparison.

1. (a)

   Source point collection

   Each reconstruction is converted into a source point cloud: mesh vertices for mesh workflows and Gaussian centres for 3DGS workflows. For 3DGS, a zero-shot anisotropy filter prunes highly anisotropic Gaussians before their centres become source points. A radius crop and optional voxel-hash density filter then remove distant background content, skyboxes, and isolated floating points before registration.

2. (b)

   Target point collection

   The Quest-side target cloud is built from the Meta Quest 3 Depth API. A point cloud generator raycasts screen-space samples into world-space points, while chunked storage and density caps keep the scan practical for in-headset use.

3. (c)

   Coarse registration

   After voxelisation reduces density mismatch, FPFH-based correspondences provide a global initialisation. MR-Compare supports robust coarse estimators such as TEASER++ and TurboReg, producing an initial alignment between reconstruction and headset scan.

4. (d)

   Fine registration

   The coarse transform is refined with G-ICP or V-GICP through the small_gicp backend. The resulting transform is persisted in the Quest world coordinate system so the aligned asset remains spatially grounded during comparison.

5. (e)

   XR interactive comparison

   The final interface combines the registered reconstruction with live video see-through. A geometry-driven 3D Slider uses a movable volumetric alpha mask, letting users reveal reconstruction and physical-world content in the same spatial frame.

The system is designed for situations where the physical room is still accessible but the user also has reconstructed digital versions of that room. MR-Compare registers each reconstruction to the live MR coordinate system, so the user can inspect where a reconstruction agrees with, deviates from, or visually complements the current physical environment.

The evaluation covers five representative reconstruction workflows: RealityScan mesh, Polycam mesh, desktop 3DGS, desktop 3DGS-MCMC, and Scaniverse 3DGS. The study combines objective registration and image-based consistency metrics with subjective ratings of perceived alignment, perceived visual consistency, usability, and cognitive workload.

A second controlled Replica evaluation complements the real-world study. It uses eight NICE-SLAM Replica indoor scenes, reconstructs each scene with 3DGS and 3DGS-MCMC, and generates Quest-like target scans from the Replica meshes to isolate registration behaviour under reproducible conditions.

In this setting, MR-Compare evaluates a zero-shot anisotropy filter for 3DGS source point selection. Sweeping the anisotropy threshold shows that moderate pruning can rescue failed registrations and reduce both translation and rotation error. The reported gains are larger for standard 3DGS, with reductions of 0.35 cm / 37.6% and 0.08 degrees / 38.7%, while 3DGS-MCMC improves by 0.12 cm / 16.9% and 0.04 degrees / 29.1%.

Evaluation figures are intentionally omitted from this version of the project page and can be added after the final paper assets are prepared.

arXiv:2607.20325 ↗

BibTeX

@misc{zhu2026mrcompare,
      title={MR-Compare: A Mixed-Reality Framework for Spatially Grounded Visual Comparison of 3D Gaussian Splatting and Mesh Reconstructions with the Physical Environment},
      author={Changrui Zhu and Ernst Kruijff and Pengju Zhang and Simon Julier},
      year={2026},
      eprint={2607.20325},
      archivePrefix={arXiv},
      primaryClass={cs.GR},
      url={https://arxiv.org/abs/2607.20325},
}