Photogrammetry vs NeRF vs Gaussian splatting

Photogrammetry, NeRF, and Gaussian splatting are three distinct methods for capturing real-world environments as 3D assets — each with a different output format, workflow cost, and fit for virtual production or game dev pipelines. At Sinfull Studios in Regina, Saskatchewan, we work with all three depending on what the shot needs: a clean editable mesh, a photoreal novel-view render, or a real-time splat cloud that drops into Unreal Engine without retopology. The right choice depends on how you plan to light, edit, and render the asset downstream.

What does photogrammetry actually produce?

Traditional photogrammetry — using software like RealityCapture or Metashape — takes a set of overlapping photos and reconstructs a textured polygon mesh. The output is geometry you can import, retopologize, UV unwrap, and bake to. Because you end up with an actual mesh and texture maps, it slots cleanly into any DCC tool or game engine. Lumen and Nanite in Unreal Engine 5 both handle photogrammetry-derived meshes well — Nanite virtualizes the polygon count and Lumen handles real-time indirect lighting. The tradeoff is capture discipline: you need controlled overlap, ideally controlled lighting, and a post-processing pipeline that goes from raw photos to clean mesh to engine-ready asset. That pipeline has real labor cost.

What is NeRF and when does it matter?

NeRF — Neural Radiance Field — is a technique that trains a neural network to represent a scene as a continuous volumetric function. Given a set of input images with known camera poses, the network learns to synthesize novel viewpoints at high quality. The outputs are photorealistic and handle view-dependent effects like reflections and translucency better than most mesh pipelines, but the representation is implicit — you cannot pull out a polygon mesh in any useful sense. NeRF is genuinely strong for previs, look development, and virtual camera work where you want to explore a captured space, but it has real limits: training takes meaningful compute time, real-time rendering of raw NeRFs has historically been expensive, and integrating the output with conventional Unreal assets requires exporting proxy geometry or baking. Luma AI, which makes the Dream Machine generative video model and earlier built 3D-capture tools on NeRF and related techniques, helped bring this workflow to a wider creative audience.

What makes Gaussian splatting different?

3D Gaussian splatting (3DGS) represents a scene as a large collection of 3D Gaussians — oriented, colored ellipsoids — rather than polygons or a neural network. The renderer rasterizes those splats from a given viewpoint, which turns out to be fast enough for real-time or near-real-time playback on modern GPUs. The visual quality is striking, particularly for complex natural environments where mesh reconstruction would produce noise or missing geometry. The method is newer than photogrammetry and is still evolving fast in the research and tooling space. The catch is the same as NeRF: the output is not conventional geometry. You cannot deform a splat cloud, apply a PBR shader to it, or drop it into a Lumen lighting rig and expect the engine to treat it like a mesh. It is a captured appearance, not editable geometry.

How does editability and relighting change the decision?

This is the practical deciding factor for most virtual production work. If the environment needs to match a lighting condition that was not present during capture — a different time of day, a colored practical light on set, integration with an LED volume’s real-time Lumen scene — you need geometry with real surface normals and PBR materials. Only photogrammetry gives you that out of the box. NeRF and Gaussian splatting bake the lighting from capture into the representation; relighting them is a research problem, not a solved production tool today. If the captured environment is purely a background plate that will never be relit and never be touched by a CG character’s shadow or reflection, the appearance-fidelity of splats or NeRF may be worth the tradeoff.

What about Unreal Engine integration?

Photogrammetry meshes integrate into Unreal the way any static mesh does — Nanite, LODs, collision, instancing, all of it. Gaussian splatting support in Unreal is available through third-party plugins and is improving, but it is still outside the core engine workflow. NeRF outputs can be baked to mesh proxies for engine import, with quality loss in complex regions. For a virtual production pipeline where the Unreal scene drives the LED volume output through nDisplay, tight engine integration matters — a splat cloud that cannot participate in the scene’s real-time GI is a liability, not a feature.

Which method fits which type of project?

  • Hero environment assets, architectural reconstruction, or anything that gets lit on set: photogrammetry. Clean mesh, PBR textures, full engine integration.
  • Previs, virtual scouting, or locked-camera background work where you just need a photoreal space to move through: NeRF or Gaussian splatting are both reasonable. Splats render faster at playback; NeRF tooling is more mature in some capture pipelines.
  • Fast turnaround background plates for a project where the camera path is constrained and relighting is not required: Gaussian splatting wins on speed-to-result. Capture a space, train, render — less retopology work than photogrammetry.
  • Complex natural surfaces — foliage, rock, weathered materials — at hero quality: photogrammetry with Nanite still gives you the most control over the final look inside the engine.

What does capture effort actually look like?

All three methods start with photographs or video, but the discipline required differs. Photogrammetry rewards methodical capture with high overlap and consistent exposure. NeRF and Gaussian splatting are more tolerant of casual video walkthroughs, which is part of why they have attracted attention for fast turnaround work. The compute side has inverted somewhat — photogrammetry reconstruction in tools like RealityCapture is fast on modern hardware, while NeRF training has historically taken longer, though newer methods and hardware have tightened that gap. Gaussian splatting training tends to be faster than NeRF for comparable scene complexity, which is one practical reason it has gained ground quickly in production-adjacent workflows.

How does Sinfull Studios approach 3D capture?

At Sinfull Studios, the default for any environment asset that will live inside an Unreal scene — on the LED volume or in a final rendered VFX shot — is photogrammetry. The mesh gives us full control over lighting, integration with CG elements, and asset reuse across projects. We watch Gaussian splatting closely because the speed-to-photoreal result is genuinely useful for specific tasks, and engine support is improving. NeRF in its pure form is less relevant to our day-to-day pipeline right now, though novel-view synthesis research continues to improve rapidly. The honest answer for most Saskatchewan and Canadian production studios is: photogrammetry for hero assets you control, splats for fast photoreal backgrounds you will not touch, and keep an eye on how engine-native 3DGS support matures over the next year or two.

Explore Environment Art in Unreal Engine at Sinfull Studios for more.

Frequently Asked Questions

Can Gaussian splatting be used in Unreal Engine for virtual production?

Gaussian splatting can be imported into Unreal Engine through third-party plugins, but it is not yet a first-class citizen in the engine’s core workflow. Splat clouds do not participate in Lumen global illumination, cannot be deformed, and do not carry PBR material data — which limits their use in virtual production setups where the scene drives an LED volume that needs real-time relighting. They work better as locked-camera background plates than as interactive environment assets.

What is the main difference between NeRF and Gaussian splatting for 3D capture?

Both NeRF and Gaussian splatting take photographs or video as input and produce a photoreal representation of a scene, but they work differently under the hood. NeRF trains a neural network to model the scene as a continuous volumetric function; rendering is slower and historically compute-intensive. Gaussian splatting represents the scene as millions of 3D ellipsoids (Gaussians) that rasterize quickly, enabling faster training and real-time or near-real-time playback. Neither produces an editable polygon mesh, which is the key practical distinction from photogrammetry.

Is photogrammetry still worth using when NeRF and Gaussian splatting exist?

Yes — for most production use cases where the asset needs to be lit, edited, or integrated with CG elements, photogrammetry is still the right choice. It produces a polygon mesh with real surface normals and PBR texture maps, which means it can be relit, deformed, collided with, and instanced inside a game engine or renderer. NeRF and Gaussian splatting bake the captured lighting into the representation, so they are difficult to relight and cannot participate in a real-time global illumination system like Unreal Engine’s Lumen. Photogrammetry’s higher capture and post-processing discipline pays off in downstream flexibility.

Related reading from Sinfull Studios

Planning a virtual production, Unreal Engine, or VFX project in Regina or anywhere in Saskatchewan? Request a quote from Sinfull Studios.