3D Capture & Reality Modeling

What is Gaussian Splatting?

3D Gaussian Splatting is a way to reconstruct a photorealistic 3D scene from ordinary overlapping photos — and render it in real time. Instead of a mesh or a point cloud, the scene is represented as millions of tiny, soft, colored 3D blobs (“Gaussians”) that blend together into a startlingly lifelike image from any viewpoint.

What
A radiance-field method that renders photorealistic novel views from photos in real time.
Introduced
2023, by a team at INRIA — now one of the fastest-moving areas in 3D graphics.
Represents a scene as
Millions of 3D Gaussians (soft ellipsoidal blobs) with position, shape, color, and opacity.
Best at
Lifelike, real-time fly-throughs — reflections, fine detail, and view-dependent shine.
A photorealistic 3D Gaussian splatting reconstruction of a collapsed building captured from drone footage
A 3D Gaussian splatting reconstruction of a collapsed building, built from drone footage — rendered as a novel viewpoint the camera never actually shot. Image: Jurdein, CC BY-SA 4.0, via Wikimedia Commons.

Radiance Fields: A Different Goal

Traditional photogrammetry asks “what is the surface here?” and outputs a mesh. Radiance-field methods ask a different question: “for any point in space, looking in any direction, what color and brightness of light would I see?” Answer that everywhere and you can synthesize a photo from any new camera position — called novel-view synthesis.

That framing captures things meshes struggle with: reflections, transparency, fuzzy materials like foliage and hair, and the way a glossy surface changes as you move. Gaussian splatting is a radiance-field method that made this both fast and practical.

What a “Splat” Actually Is

Picture a point cloud where every point is not an infinitely small dot but a small, soft, semi-transparent ellipsoid — a 3D Gaussian. Each one stores:

  • Position — where it sits in 3D space.
  • Shape — how it is stretched and oriented (a flat splat hugs a wall; a round one fills volume).
  • Color & view-dependence — its color, and how that color shifts with viewing angle (so a surface can look shiny from one side).
  • Opacity — how solid or faint it is.

To draw the scene, the renderer projects (“splats”) all these blobs onto the screen and blends them back-to-front. Millions of overlapping splats add up to a continuous, photorealistic image — and because this is just sorting and blending, modern GPUs do it at 100+ frames per second.

How a Scene Is Built

The capture input is familiar — the same kind of overlapping photos or video you would shoot for photogrammetry. From there:

  1. Camera poses are recovered from the photos (the same Structure-from-Motion step photogrammetry uses), giving a sparse starting point cloud.
  2. That cloud is seeded with 3D Gaussians.
  3. Training renders the splats, compares the result to the real photos, and nudges every Gaussian’s position, shape, color, and opacity to reduce the difference — repeating millions of times.
  4. The optimizer adds detail where the scene needs it and prunes splats that do not help, until the rendered views match the photographs.
Diagram of the 3D Gaussian splatting training and rendering pipeline
The training loop: 3D Gaussians are projected and blended into an image, compared against the real photos, and refined — densifying detail where it is needed. Image: Kerbl, Kopanas, Leimkühler & Drettakis (INRIA), CC BY 4.0, via Wikimedia Commons.

Gaussian Splatting vs NeRF

NeRF (Neural Radiance Fields), introduced in 2020, was the method that put radiance fields on the map. It stores the scene inside a neural network: to find the color at a point you query the network. The results were groundbreaking, but querying a network for every pixel made the original NeRF slow to train and far too slow to render in real time.

Diagram of the NeRF pipeline turning input images into a neural radiance field and novel rendered views
The NeRF pipeline: input photos train a neural network that, queried along rays, renders novel views. Image: Mildenhall et al., MIT License, via Wikimedia Commons.

Gaussian splatting reaches similar or better visual quality but stores the scene explicitly as blobs rather than implicitly in a network. That one change is decisive:

NeRF

  • Scene lives in a neural net
  • Slow to render (originally)
  • Harder to edit directly

Gaussian Splatting

  • Scene is explicit blobs
  • Real-time rendering (100+ fps)
  • Faster to train, easy to view

Both

  • Radiance fields from photos
  • Great with reflections & fuzz
  • Novel-view synthesis

vs Traditional Photogrammetry

They start from the same photos but aim at different deliverables. Photogrammetry gives you a clean, measurable mesh and point cloud — engineering-grade geometry you can take measurements off. Gaussian splatting gives you a stunning, view-realistic experience — better-looking fly-throughs, but the “surface” is a fuzzy cloud of blobs rather than crisp, directly-measurable geometry.

★

Looks vs measurement

Need to measure a beam, compute a volume, or feed CAD? Reach for photogrammetry or LiDAR. Need the most lifelike walkthrough of a space, including glass and shine? Gaussian splatting is hard to beat. The two are increasingly used together — accurate geometry for measurement, splats for presentation.

Where It Fits

  • Visualization & presentation — immersive, photoreal fly-throughs of sites, properties, and as-built conditions.
  • Digital twins & heritage — capturing a place exactly as it looked, reflections and all.
  • VFX, AR/VR, and games — dropping real captured environments into real-time engines.
  • Robotics & mapping research — a fast, photoreal scene representation that is being actively extended.

Gaussian splatting is young and moving fast, but it has already changed what “capturing reality” can look like — turning a phone video into a 3D scene you can walk through. Pair it with the measurable outputs of LiDAR and photogrammetry and you have the full reality-capture toolkit.