Android XR headset: tool baseline¶
Which existing tools work on the Android XR headset emulator, and what limits remain. The coordinate model, the shared XR predicate and the routing rules are in Android form factors; this page is the tool-by-tool result.
Setup¶
- Image
system-images;android-34;google-xr;arm64-v8a, profilexr_headset_device(2560×2558 display, 2558×2558 composed screenshot), classifiedxr-headset. - CtrlProxy built from the same checkout as the daemon. A cached released CtrlProxy APK reports the Settings rows as hidden on this image, so results only hold with the current CtrlProxy build.
- Targets: Settings (the 2D panel used for most rows), Chrome, Play Store and a sideloaded sample APK. No app that draws 3D (SceneCore) content was exercised.
- Every result is at the default headset pose in home space; the panel was never moved and no immersive layout was opened.
Matrix¶
| Tool | Result | Limitation |
|---|---|---|
startDevice (existing AVD, cold boot) |
pass | Reaches automationReady in about 30 s; CtrlProxy installs and its accessibility service is enabled by startDevice. |
killDevice |
pass | Needs device: {name, deviceId, platform} plus the holder’s sessionUuid; a top-level deviceId is a validation error that names the right shape. |
observe, app panel |
pass | Panel rows are listed with text. Embedded SpatialPanel observations identify the window in xrPanelSpace; element and skeleton/context bounds carry boundsSpace: "panel" or "window" for other windows. Panel bounds are window-relative, not screen pixels; other windows do not share that space. |
observe, home space |
partial | Lists the System UI XR windows with bounds in their own window space (larger than the screen), plus hidden System UI nodes that are not drawn (for example the “Unable to detect surroundings” dialog). accessibilityState reports enabled: false, service: "unknown" while the service is bound. |
observe screenSize after the keyboard shows |
pass | Stays 2560×2558 before, during and after the keyboard shows, and nodes below y=608 stay in the capture. |
observe screenshot |
pass | The composed view: passthrough environment, 2D panels and the keyboard. 3D (SceneCore) content is untested. |
tapAt |
pass | Takes screenshot pixels and maps them to the panel at the default pose only; the result carries an xrPanel block and a default-pose warning. Works with Gboard shown. |
tapOn |
pass | Clicks the accessibility node for panel rows (“Apps”, “Network & internet”, “Connected devices”, “Storage”, “Search”). Hidden System UI chrome (“Environment”, “Recents”, “Clear all”) is refused with guidance to use observe or tapAt. While a dialog window of the same app is open, a row beneath it is refused. |
tapAny |
pass | With no container it taps a panel control; a container named by a leaf text (“Apps”) resolves to its clickable row; a scrollable container taps its first fully visible row. While a dialog window of the same app is open, it picks a dialog control. |
swipeOn |
pass | A container swipe scrolls the panel list and keeps the app resumed. A swipe that would leave the app panel is refused. On a Compose SpatialPanel it scrolls the container, or else the largest visible list, with accessibility scroll actions, and is refused when there is no list. An unscoped swipe on a non-scrollable pane scrolls nothing and dismisses nothing. |
sendKeys type, Auto |
pass | Types through the AutoMobile IME (resolvedMode: ime). The Auto route to a11y when Gboard is the shown keyboard is untested on the device; an explicit mode: "a11y" types into the focused field. The result warning names the route taken. |
pressButton back, home |
pass | Back first closes the keyboard, then pops the activity. Home moves focus to the System UI visibility barrier. |
launchApp |
pass | The app opens as a floating 2D panel; foreground is verified. |
listApps |
pass | Lists launchable apps, including the OpenXR sample. |
terminateApp |
pass | None. |
installApp, uninstallApp |
pass | None. |
getLogs |
pass | Works although startDevice lists it in gatedTools. |
videoRecording start and stop |
pass | h264 MP4 at 2558×2558 showing the composed environment. |
| Head pose or view change | fail | No console or adb surface: adb emu physics only records ground truth, adb emu sensor set orientation leaves the screenshot unchanged, and cmd xrsimulatedinputmanager has no shell implementation (#16843). |
Not covered¶
- 3D (SceneCore) content in
observe, screenshots and input. - Moved panels, more than one app panel, and immersive layouts.
- XR glasses (
ai-glassesimages).