TL;DR:

  • Photogrammetry reconstructs 3D geometry and texture from overlapping photographs — no depth sensor required, just a camera and good capture discipline
  • Modern pipelines (RealityCapture, Metashape, Polycam) can go from photos to a textured mesh in under an hour for objects; larger scenes take longer
  • Output quality depends almost entirely on capture quality: consistent lighting, high overlap, and no reflective or translucent surfaces

What Photogrammetry Produces

Photogrammetry takes a set of overlapping photographs of an object or environment and uses computer vision to:

  1. Identify matching features across images (tie points)
  2. Reconstruct camera positions relative to each other (structure from motion)
  3. Build a dense point cloud from pixel depth estimates
  4. Convert the point cloud to a polygon mesh
  5. Project the source photographs onto the mesh as a texture

The result is a textured 3D model that can be exported in formats compatible with XR engines — GLB, glTF, USD, FBX, or OBJ depending on pipeline.

Photogrammetry is distinct from LiDAR scanning (which uses time-of-flight laser pulses to measure distances directly) and from newer neural reconstruction methods like NeRF and Gaussian splatting (which represent scenes as learned fields rather than meshes). All three produce 3D representations from real-world capture, but their workflows and output characteristics differ significantly.

When to Use Photogrammetry vs Alternatives

Photogrammetry works well for:

  • Objects with diffuse, textured surfaces (stone, wood, fabric, rough metal, food, props)
  • Environments and architectural spaces
  • Situations where you have only a camera phone or DSLR, no depth sensor
  • Cases where you need an exportable polygon mesh for engine import

Photogrammetry struggles with:

  • Reflective surfaces (chrome, glass, glossy plastic) — the appearance changes with view angle, breaking the feature matching
  • Translucent materials — light passes through, confusing depth estimates
  • Plain, featureless surfaces — feature detectors need texture to find tie points
  • Moving subjects — the object must be completely still throughout capture

LiDAR scanning (iPhone Pro LiDAR, Matterport, FARO terrestrial scanners) is faster for room-scale capture and handles featureless surfaces better, but produces coarser textures than photogrammetry. For large indoor environments, a combined approach works well: LiDAR for geometry, photogrammetry textures projected onto the LiDAR mesh.

Gaussian splatting (using tools like Luma AI, Polycam’s Gaussian mode, or open-source implementations of 3D Gaussian Splatting) produces visually impressive results but outputs a splat representation rather than a mesh. Splats are not universally supported in XR engines yet — Unity and Unreal have plugins, but native glTF export with full viewer compatibility is still limited. For near-term XR production, mesh-based photogrammetry remains more practical.

Capture Workflow

Object Capture

For an object up to roughly desk-sized:

Setup: Place the object on a plain, matte surface (not white — a mid-grey or dark surface with some texture helps the reconstruction). Shoot in diffuse light: overcast daylight, a lightbox, or soft artificial lighting from multiple directions. Avoid harsh shadows. If the surface is shiny, consider a temporary matte spray coating (artist fixative works and is reversible).

Camera circuit: Walk around the object taking photos at 10-20 degree intervals. Aim for 60-80% overlap between adjacent images. Do multiple passes at different heights: one circuit at object-level, one at 45 degrees above, and one near-overhead. For complex objects with undercuts or recessed areas, add extra passes focusing on those areas.

Photo count: 60-150 images is typical for a medium-complexity object. More is not always better — redundant images add processing time without improving quality.

Settings: ISO as low as practical (ISO 100-400), fast enough shutter to avoid motion blur (1/100s or faster if handholding), aperture around f/8 for depth of field. RAW gives you more latitude in post but JPEGs at maximum quality work fine for most workflows.

Environment Capture

For rooms, architecture, or outdoor spaces:

Overlap is critical. Every area of the scene must appear in at least three photos from significantly different positions. Walk in systematic patterns rather than random paths.

For interiors: Shoot from multiple heights if possible. Doorways and corners are often problematic — give them extra attention.

For exteriors: Time of day matters significantly. Midday sun creates harsh shadows that confuse reconstruction; overcast or golden hour light is better. For large buildings, drone capture from multiple altitudes and angles dramatically improves completeness.

Software Options

RealityCapture (Epic Games, ~€3,750/year enterprise or PPL pricing): The fastest photogrammetry software available. Handles large datasets (thousands of images) in hours rather than days. Output quality is excellent. The per-image pricing model is useful for occasional use.

Agisoft Metashape (~$179 standard, $3,499 professional): The most widely used in professional production. Slower than RealityCapture on large datasets but very reliable. Good batch processing, Python API for automation, and long track record in film, VFX, and architecture.

Polycam (mobile, freemium): Runs on iPhone and iPad. Uses LiDAR for room-scale capture and photogrammetry for object capture. The fastest path from phone to 3D model for quick scans. Quality is adequate for prototyping and background assets; not production-ready for hero objects.

Meshroom (free, open source): Based on AliceVision. Slower than commercial options but free. Good for learning the pipeline; less practical for production use at scale.

Apple Object Capture API: Built into macOS and iOS, uses the RealityKit framework. If your target is Apple Vision Pro or ARKit, this produces USDZ output natively. Requires iPhone 12 Pro or later for the LiDAR-assisted mode; works without LiDAR for pure photogrammetry.

Processing and Cleanup

Raw photogrammetry output usually needs cleanup before it’s usable in an XR scene:

Mesh cleanup: Remove floating geometry, background capture, and reconstruction artefacts. Blender’s sculpting tools and the Decimate modifier are common choices. The mesh will often be denser than necessary — decimation reduces polygon count while preserving visual quality.

UV unwrapping and texture baking: Photogrammetry meshes come with projection-based textures tied to the capture photos. For production assets, rebaking to a clean UV layout improves compression and runtime performance. Tools: Blender, Substance Painter, Marmoset Toolbag.

LOD generation: XR applications need multiple level-of-detail versions. A hero asset might use 50k triangles at close range and 2k at distance. Generate LODs using Simplygon, Blender’s Decimate, or Unreal Engine’s built-in LOD tools.

Normal map baking: Baking high-frequency detail from the dense mesh onto a normal map for the lower-poly production mesh recovers surface detail that would otherwise be lost in decimation.

Integration with XR Engines

Unity: Import GLB or glTF directly. Verify that the Universal Render Pipeline (URP) or High Definition Render Pipeline (HDRP) is configured appropriately for the target platform (Quest uses URP).

Unreal Engine: Nanite handles high-poly meshes efficiently at runtime, so heavy mesh decimation is less critical than in Unity. Import as FBX or GLB. Use Unreal’s automatic LOD generation if needed.

Apple Vision Pro / visionOS: USDZ is the native format. Apple’s Reality Composer Pro handles import and scene composition. For photogrammetry captured with Object Capture, the pipeline is native.

Meta Quest: GLB with optimised textures (KTX2 with Basis Universal compression). Poly count and texture resolution directly affect performance — target well under 100k triangles for interactive objects on Quest 3.

Practical Expectations

A medium-complexity object shot in good conditions with 80-100 photos, processed in Metashape or RealityCapture, will produce a usable production asset in a couple of hours including processing time. The captured texture will look realistic precisely because it is — photogrammetry’s main advantage over hand-authored 3D models.

Environments take longer, have more cleanup, and often benefit from manual retopology for characters and hero assets. For architectural walkthroughs, hero-object detail alongside lower-quality environment meshes is a common production pattern.

The technique’s main failure mode is not technical — it’s capture discipline. Good software cannot recover from poor overlap, blown highlights, or a subject that moved between frames. Investing time in the capture process pays off far more than experimenting with processing settings.