The fundamental problem with high-resolution VR is arithmetic. A human field of view spans roughly 200 degrees horizontally; current headsets cover 100–120 degrees and render everything in that frame at full resolution. Covering that area at the pixel density required to avoid screendoor effect — roughly 4K per eye for convincing fidelity — demands GPU performance that either requires a tethered PC or produces heat and battery drain that makes standalone headsets impractical.

Eye tracking changes the constraint. The human eye has high-resolution vision only in the fovea — a tiny 2-degree area at the centre of your gaze. Everything outside that region is perceived at progressively lower resolution. Your visual cortex fills in the gaps so convincingly that you’re normally unaware of this. A rendering system that knows where you’re looking can exploit this: render the foveal region at full resolution and drop peripheral resolution significantly, with no perceived quality difference.

This is foveated rendering, and when it’s paired with accurate real-time eye tracking, it cuts rendering workload dramatically enough to change what’s possible on standalone hardware.

How eye tracking works in headsets

Modern headset eye tracking uses infrared LED illuminators and near-infrared cameras mounted inside the headset, pointed at the eyes. The LEDs project patterns onto the corneal surface; the cameras capture the reflected patterns and the position of the pupil. Software combines these signals to triangulate gaze direction in real time.

The output is typically a 3D gaze vector — where in 3D space the user is looking — updated at 90–120 Hz to match display refresh rates. Most production implementations also output confidence values, useful for filtering out blinks and low-light tracking failures.

Accuracy varies significantly by implementation:

  • Consumer headsets (Meta Quest Pro, PS VR2): 1.5–3 degrees accuracy, which is sufficient for foveated rendering and basic UI gaze interaction
  • High-end professional headsets (Varjo XR-4, HTC VIVE Pro Eye): 0.5–1 degree accuracy, enabling research-grade gaze data collection and more precise UI targeting
  • Apple Vision Pro: reported 1–2 degree accuracy with the additional sophistication of using eye tracking as the primary pointing input for the entire UI

Dynamic foveated rendering in practice

Standard fixed foveated rendering (FFR) — rendering a fixed inner region at full resolution — has been available since Meta Quest 2. Eye-tracking-enabled dynamic foveated rendering (ETFR) goes further: the high-resolution region follows your gaze in real time.

The GPU gains are substantial. Varjo has published figures showing 40–60% reduction in GPU workload with ETFR enabled at equivalent perceived quality. Meta’s internal benchmarks for Quest Pro showed similar results. The practical impact:

  • Higher visual fidelity on fixed GPU budgets — the power saved can go into resolution, anti-aliasing, or scene complexity
  • Lower thermal load — relevant for standalone headsets where heat dissipation limits sustained performance
  • Better battery life — lower GPU utilisation directly extends runtime
  • Lighter headsets — if designers can assume ETFR, they can spec a less powerful GPU with the same effective visual quality

Apple Vision Pro uses this aggressively: the display renders at full resolution only in the gaze region, with peripheral areas rendered at lower resolution. The system is calibrated during setup specifically to map the relationship between eye position and rendered foveal region for each user’s anatomy.

Current hardware landscape

Eye tracking is no longer rare in premium headsets, but it’s absent from most mass-market standalone devices:

Apple Vision Pro: Eye tracking is the primary input mechanism — you look at something and pinch to select it. Exceptionally accurate, calibrated per-user at setup. Used for foveated rendering and all UI interaction.

Meta Quest Pro: Eye and face tracking included. Used for avatar expression (social presence features) and available to developers via OpenXR extensions for ETFR. Not present in Quest 3 or Quest 3S, which use fixed foveated rendering instead.

PlayStation VR2: Eye tracking included, used for foveated rendering. Sony has published developer guidance on accessing gaze data for ETFR. Some games (Gran Turismo 7, Horizon: Call of the Mountain) implement ETFR on PS VR2.

Varjo XR-4 and VR-3: High-accuracy eye tracking (0.5 degree), used for foveated rendering and gaze data export for research. These are professional headsets; the eye tracking quality enables use cases like attention research and surgical training where accuracy matters.

HTC VIVE Pro Eye: Earlier high-accuracy eye tracking, still used in research settings. More recent VIVE products have deprioritised eye tracking in consumer configurations.

Pico 4 Ultra (ByteDance): Eye tracking added in 2024 update. Used for foveated rendering and social avatar features.

Developer implementation

For XR developers targeting eye-tracked platforms, the access point is the OpenXR eye gaze interaction extension (XR_EXT_eye_gaze_interaction) for reading gaze vectors, and vendor-specific extensions for configuring foveated rendering:

Unity:

using UnityEngine;
using UnityEngine.XR;

public class EyeGazeManager : MonoBehaviour
{
    private InputDevice leftEye;
    private InputDevice rightEye;

    void Start()
    {
        leftEye = InputDevices.GetDeviceAtXRNode(XRNode.LeftEye);
        rightEye = InputDevices.GetDeviceAtXRNode(XRNode.RightEye);
    }

    void Update()
    {
        if (leftEye.TryGetFeatureValue(CommonUsages.eyesData, out Eyes eyesData))
        {
            if (eyesData.TryGetFixationPoint(out Vector3 fixationPoint))
            {
                // fixationPoint is the 3D world-space fixation point
                // Use this to direct rendering resources or drive gaze-based UI
                Debug.DrawRay(Camera.main.transform.position, 
                    fixationPoint - Camera.main.transform.position, Color.red);
            }
        }
    }
}

Foveated rendering in Unity is configured via the XRRenderingExtensions API or through the Meta XR SDK’s OVRManager component, which exposes foveatedRenderingLevel settings. The platform SDK handles the actual per-frame rendering adjustment.

Unreal Engine exposes eye tracking through the EyeTracker plugin and GazeData struct. Epic’s OpenXR plugin supports the XR_EXT_eye_gaze_interaction extension, and foveated rendering is configured through the Variable Rate Shading (VRS) pipeline.

Gaze interaction beyond rendering

Eye tracking enables interaction patterns that are significant for enterprise use cases:

Dwell-based selection: looking at a UI element for a configurable duration triggers selection without any physical button press. Useful for hands-free workflows in surgery, manufacturing, or cleanroom environments where users wear gloves or hold tools.

Attentional analytics: in training simulations, knowing precisely where a trainee looked during a procedure provides data unavailable through any other sensor. Medical training companies like Oxford Medical Simulation and Osso VR use gaze data to identify whether trainees are monitoring the correct anatomical structures during simulated procedures.

Accessibility: for users with motor impairments, gaze-based interaction can replace or supplement hand tracking and controllers entirely. Apple Vision Pro’s gaze + pinch model was designed partly with accessibility as a first-class consideration.

Adaptive UI: elements can highlight when the user looks at them, pop-up descriptions can appear on sustained gaze, and UI density can dynamically adjust based on where the user’s attention is concentrated.

Privacy considerations

Eye tracking generates sensitive biometric data. Gaze patterns reveal attention, emotional state, cognitive load, and — in some research — can be used to infer neurological conditions. The regulatory picture is evolving:

  • GDPR / UK GDPR: eye tracking data is biometric data under Article 9, a special category requiring explicit consent and legitimate purpose
  • BIPA (Illinois, US): Illinois law requires written consent and specific data handling for biometric identifiers including eye scan data
  • Apple’s approach: Vision Pro processes gaze data entirely on-device; Apple’s APIs give apps the results of gaze interaction (what was selected) but not the raw gaze vectors, limiting what developers can collect

For enterprise deployments, data minimisation is the principle: collect gaze data only when it serves a specific, documented purpose (training analytics, accessibility), retain for the minimum necessary period, and store separately from identity data where possible.

What to watch

Eye tracking is moving from premium differentiator to expected feature in enterprise XR over the next two to three years. The cost reduction in capable cameras and IR illumination is significant — the hardware that enabled Varjo’s 0.5-degree accuracy in 2021 now fits in headsets at much lower price points.

The next significant development is likely full-field gaze accuracy — calibrating eye tracking not just for the centre of the display but across the full field of view, which matters for peripheral UI interactions and is necessary for the most aggressive ETFR implementations. Several research groups and at least one major headset manufacturer are actively working on this, with production implementations expected in 2026–2027 headsets.