TL;DR:

  • Reality Composer Pro is Apple’s standalone 3D scene editor for composing spatial content; RealityKit is the runtime framework that renders and animates it in your visionOS app
  • The development workflow: design scenes in Reality Composer Pro, export as .reality bundles or .usdz files, load them in your SwiftUI/RealityKit code with Entity.load()
  • RealityKit uses an Entity Component System (ECS) architecture — entities are containers, components add behaviour (physics, audio, animations), systems process them each frame

If you’re coming to visionOS development from iOS or macOS, RealityKit is probably the biggest new concept to absorb. It’s not UIKit or SwiftUI — it’s a 3D rendering framework with its own scene graph, a different mental model for organising content, and tight integration with Reality Composer Pro for content authoring.

The good news is that the integration between Reality Composer Pro and RealityKit is genuinely well thought out, and once the model clicks, building 3D content for visionOS feels relatively natural.

Reality Composer Pro: What It Is

Reality Composer Pro (RCPRO) is a standalone macOS application (separate from Xcode, though tightly integrated with it) for building 3D scenes. You compose 3D content, apply materials, set up animations, configure physics, add audio, and arrange entities in a scene hierarchy. The output is a .reality file that your visionOS app loads at runtime.

Think of it as the equivalent of Interface Builder for 3D spatial content. You do the visual composition and property configuration in RCPRO, and the resulting file is compiled into your app’s asset bundle.

What RCPRO handles particularly well:

ShaderGraph materials: RCPRO includes a node-based material editor similar to Blender’s Shader Editor or Unreal’s Material Editor. You can create procedural materials, add texture effects, and write custom ShaderGraph nodes without leaving the tool.

Behaviours and timelines: You can define trigger-response behaviours (when this entity is tapped, play this animation) without writing code. For simple interactive experiences, this can eliminate a lot of boilerplate.

Physics simulation: Rigid body physics, collision shapes, and joints are configured visually and carry over directly to the runtime.

Particle systems: RealityKit’s particle system is authored in RCPRO, not in code, which makes iterating on visual effects much faster.

RealityKit’s Entity Component System

The runtime architecture is ECS (Entity Component System), which is different from the class hierarchy approach in UIKit.

  • Entity: A container in the scene graph. Has a transform (position, rotation, scale). Can have child entities.
  • Component: A piece of data attached to an entity that gives it behaviour or properties. ModelComponent renders a 3D mesh. PhysicsBodyComponent makes it participate in physics. AudioLibraryComponent attaches audio resources. You add components to entities to give them capabilities.
  • System: Code that processes all entities with a specific set of components each frame.

A minimal example — loading a scene from a Reality file and placing it:

import RealityKit
import SwiftUI

struct ImmersiveView: View {
    var body: some View {
        RealityView { content in
            // Load a .reality bundle from the app's bundle
            if let scene = try? await Entity.load(named: "SolarSystem", in: Bundle.main) {
                content.add(scene)
            }
        }
    }
}

For more control over placement and interaction:

RealityView { content in
    let anchor = AnchorEntity(.plane(.horizontal, classification: .floor, minimumBounds: [0.5, 0.5]))
    
    if let model = try? await Entity.load(named: "Table") {
        model.position = [0, 0, -1.5]  // 1.5m in front of origin
        model.generateCollisionShapes(recursive: true)
        anchor.addChild(model)
    }
    
    content.add(anchor)
} update: { content in
    // Called when SwiftUI state changes
}

Handling Gestures and Interaction

visionOS input (look + pinch) is handled through SwiftUI’s gesture modifiers applied to RealityKit entities. For tap:

RealityView { content in
    // setup
}
.gesture(TapGesture().targetedToAnyEntity().onEnded { value in
    let entity = value.entity
    // entity was tapped -- add a pulse animation, change colour, etc.
    var transform = entity.transform
    transform.scale = [1.2, 1.2, 1.2]
    entity.move(to: transform, relativeTo: entity.parent, duration: 0.1)
})

For drag:

.gesture(
    DragGesture()
        .targetedToAnyEntity()
        .onChanged { value in
            value.entity.position = value.convert(value.location3D, from: .local, to: .scene)
        }
)

The .targetedToAnyEntity() modifier is key — it makes the gesture apply to RealityKit entities with collision shapes, which is how visionOS hit-testing works.

The Immersive Space API

For full immersive experiences (filling the user’s view rather than presenting a window), visionOS has the ImmersiveSpace API:

@main
struct MyApp: App {
    @State private var immersiveSpaceIsShown = false
    
    var body: some Scene {
        WindowGroup {
            ContentView(immersiveSpaceIsShown: $immersiveSpaceIsShown)
        }

        ImmersiveSpace(id: "ImmersiveView") {
            ImmersiveView()
        }
        .immersionStyle(selection: .constant(.mixed), in: .mixed)
    }
}

Immersion styles on visionOS range from .mixed (content floats in the real world, passthrough still visible) to .full (complete virtual environment, passthrough disabled). The mixed mode is the most common for enterprise and productivity applications; full immersion is more typical for games and training simulations.

Practical Tips for the Workflow

Iterate in Reality Composer Pro first: It’s much faster to adjust a scene, material, or animation in RCPRO and see the result in the Simulator than to make the change in code. Get the visual composition right in RCPRO before adding code complexity.

Use USDZ for simpler assets: For single objects without complex behaviours, .usdz files load directly with Entity.load() and are easy to source from Apple’s USDZ library, Sketchfab, or your 3D artist. Reserve .reality bundles for multi-entity scenes with configured behaviours.

Anchor carefully: How you anchor content (to a horizontal plane, to the user’s head, to a world position) determines whether it feels like it belongs in the space. Anchoring to planes makes AR content feel grounded; head anchors are good for HUD-like UI; world anchors at specific coordinates are right for persistent experiences.

Profile early: RealityKit can be GPU-intensive. The Reality Composer Pro Performance Analyser and Xcode’s GPU Frame Capture show you polygon counts, draw calls, and shader complexity. Budget for this before you’ve built the whole experience.

The learning curve for visionOS development is real — it’s a new paradigm even for experienced Apple developers. But the toolchain is cohesive, the documentation has improved substantially with visionOS 26, and the developer community around it has grown enough that most common patterns have example code available.