GenXR-Bench
Benchmarking Coding Agents for Interactive XR Development

William Huang1 David Li2 Xiang 'Anthony' Chen1 Xun Qian2 Nels Numan2 Zhongyi Zhou2 Hyunsung Cho3 Salman Muin Kayser Chishti4 Ding Xia5 Xincheng Huang6 Jiahao Ren2 Faraz Faruqi7 Tianyu Xu2 Yang Zhang1 David Kim2 Ruofei Du2
1 University of California, Los Angeles 2 Google XR Labs 3 Carnegie Mellon University 4 University of Oxford 5 The University of Tokyo 6 University of British Columbia 7 Massachusetts Institute of Technology
↗   Paper </>   Code   XR Blocks { }   BibTeX

112 tasks · 24 SDK features · 21 SDK repairs · 67 app generation

Leaderboard

September 2026

Benchmark Results

Download CSV ↓
Benchmark results. Select a column heading to sort, or select a row to highlight in the graph.

Pass@1 is the percentage of tasks with all required assertions passing. Tokens are reported in millions.

Abstract

Coding agents are rapidly advancing from single-file generation to long-horizon repository-scale engineering, yet the evaluation of their capabilities to author interactive extended reality (XR) applications remains an open challenge. Unlike traditional software, XR applications operate within dynamic 3D physical environments where runtime robustness depends jointly on continuous user motion, spatial input, and environmental sensing. We introduce GenXR-Bench, the first benchmark for the end-to-end evaluation of coding agents in XR software development. GenXR-Bench contains 112 expert-authored tasks spanning 24 feature implementations, 21 bug repairs, and 67 application generation tasks. Held-out tests combine scripted and adaptive interactions with programmatic and visual assertions to evaluate core functionality and runtime behavior. GenXR-Bench’s testing suite achieves 96.5% accuracy against blinded expert judgments of 124 candidate applications executed on consumer XR hardware. Evaluating 10 models on GenXR-Bench, we find that the strongest configuration achieves only 50.9% overall Pass@1. Failure analysis identifies recurring misunderstandings in asynchronous sensor lifecycles, physical grounding, and embodied interaction semantics. GenXR-Bench establishes a reproducible testbed for evaluating coding agents in the interactive and physically situated conditions of spatial computing.

Contributions

  • GenXR-Bench: An open-source benchmark of 112 expert-authored tasks evaluating coding agents on XR software engineering, spanning SDK-level feature development, bug repair, and end-to-end application synthesis.
  • GenXR-Test: XR tooling for automated evaluations using scripted kinematic actors, adaptive VLM agents, runtime state assertions, and scoped visual checks in simulated 3D environments.
  • Hardware-Grounded Validation: A validation study across 372 physical XR device trials demonstrating that simulator-based evaluation achieves 96.5% agreement with expert human judgment with low flakiness (4.0%).
  • Empirical Analysis of Coding Agents in XR: An evaluation of 10 model configurations identifying key bottlenecks in frontier models, including coordinate-frame handling, sensor lifecycles, and embodied interaction semantics.

Generated Application Demos

Open in new tab ↗

On-Device Examples

Recorded directly from Galaxy XR. Demos generated with Google Gemini 3.7 Flash.

Citation

@misc{huangGenXRBenchBenchmarkingCoding2026,
    title = {{{GenXR-Bench}}: {{Benchmarking Coding Agents}} for {{Interactive XR Development}}},
    shorttitle = {{{GenXR-Bench}}},
    author = {Huang, William and Li, David and Chen, Xiang 'Anthony' and Qian, Xun and Numan, Nels and Zhou, Zhongyi and Cho, Hyunsung and Chishti, Salman Muin Kayser and Xia, Ding and Huang, Xincheng and Ren, Jiahao and Faruqi, Faraz and Xu, Tianyu and Zhang, Yang and Kim, David and Du, Ruofei},
    year = 2026,
    month = oct,
    number = {arXiv:2610.29014},
    eprint = {2610.29014},
    primaryclass = {cs.AI},
    publisher = {arXiv},
    doi = {10.48550/arXiv.2610.29014},
    urldate = {2026-10-22},
    archiveprefix = {arXiv}
}