๐งฐ Tools¶
Every tool works from the CLI (bunx @kaeawc/auto-mobile --cli <tool>), MCP, or the daemon HTTP endpoint โ same names, same arguments.
๐ Full arguments: the registered schema (schemas/tool-definitions.json) or automobile:tools. Availability varies by platform and feature gates: see the Platforms and Needs columns.
๐ Platforms and Needs columns¶
Each tool’s _meta["automobile/requires"] list, shown as emoji in the tables below: Platforms lists the devices the tool drives, Needs lists what to turn on first. Hover an emoji for its name. On a phone, tap Show descriptions above a table to add the What it does column.
| Emoji | Column | Requirement | Meaning |
|---|---|---|---|
| ๐ค | Platforms | platform:android |
the tool drives Android devices |
| ๐ | Platforms | platform:ios |
the tool drives iOS devices |
| โ | Platforms | platform:watchos |
the tool drives watchOS devices |
| ๐บ | Platforms | platform:tvos |
the tool drives tvOS devices |
| ๐ฅฝ | Platforms | platform:visionos |
the tool drives visionOS devices |
| ๐งฉ | Needs | sdkEmbed |
start the daemon with --embedded-sdk |
| ๐ | Needs | debugBuild |
start the daemon with --debug (the debug feature flag) |
| ๐ฉ | Needs | featureFlag:<key> |
some or all calls are refused while that feature flag (named beside the emoji) is off; enable it with auto-mobile --daemon set-feature-flag <key> true |
- A tool may still answer
unsupportedfor an action its description limits to one platform. - A
โPlatforms cell: a host tool that needs no device. AโNeeds cell: nothing to turn on.
๐ท๏ธ Tool discovery metadata¶
- ๐
tools/listentries carry a human-readabletitleand SDKToolAnnotations. - ๐ก๏ธ Hints are untrusted client guidance, never authorization.
- ๐๏ธ Classification lives in
src/server/toolAnnotations.ts. - ๐ Mixed-action tools take the conservative union of all supported actions.
- ๐ฆ The wire omits default
readOnlyHint: false,idempotentHint: false, andopenWorldHint: true. - ๐ Read-only tools omit the meaningless destructive and idempotent hints.
- ๐๏ธ Controls always declare
destructiveHintexplicitly. - ๐ Metadata is independent of
deviceReadOnlydispatch and ownership enforcement. - ๐งช
readOnlyHintfollowsdeviceReadOnly, except tools that also write host state (compareScreenshotrecord mode).
๐ Shared options¶
- ๐
sessionUuidโ run on a held device session (device tools) - ๐ก
keepScreenAwakeโ wake and keep a physical Android device awake (default on at session setup) - ๐ช
raw/project(skeletondefault,full) /includeExtrasโ observation output on action tools - ๐
resourceLinkโgetLogs,exportPlan, andvideoRecordingstop: return the complete result as anautomobile:tool-outputresource and aresource_linkblock, keeping onlysuccess,error, and the result’s required fields (a stop’saction,count,recordings, each bounded) inline (default off, inline) - ๐ฑ
deviceIdโ target a device explicitly on reads
Reading a device you do not own (deviceId)¶
A read never writes settings on a device another session holds. A sessionless read of an unheld running device may ready it (install CtrlProxy / runner) to answer. See which read tools ready an unheld device.
๐ Observe & navigate¶
| Tool | What it does | Platforms | Needs |
|---|---|---|---|
๐ธ snapshotOf |
Save a PNG crop of one exposed element or a platform-native screen rectangle | ๐ค ๐ | โ |
๐ compareScreenshot |
Visual regression: record a named screenshot baseline or compare the current screen, an element, or a rectangle against it, ignoring masked regions and system bars | ๐ค ๐ ๐บ | โ |
๐๏ธ observe |
Get screen view hierarchy and screenshot, with optional PNG crop from a fresh settled capture of one display | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐ฏ hitTest |
Preview hierarchy-bounds candidates at a platform-native screen point | ๐ค ๐ | โ |
๐งญ explore |
Automatically interact with the app to discover screens and record transitions in the navigation graph | ๐ค ๐ ๐บ | ๐งฉ |
๐บ๏ธ navigateTo |
Navigate to a learned screen by replaying recorded transitions from the navigation graph | ๐ค ๐ | ๐งฉ |
๐ธ๏ธ getNavigationGraph |
Get the learned navigation graph for the foreground or given app | ๐ค ๐ | ๐งฉ |
๐ identifyInteractions |
Suggest likely interactions | ๐ค ๐ | ๐ |
๐งช prototype |
Show agent-authored UI on the device | ๐ค ๐ | โ |
๐ป exportPrototypeWeb |
Write the automobile:prototype/<id>/web page to an HTML file |
โ | โ |
๐ก blePeripheral |
Manage a session-owned virtual BLE peripheral on Android netsim | โ | โ |
๐ฆ highlight |
Draw a red hand-drawn circle around a UI element | ๐ค ๐ | โ |
๐ Interact¶
| Tool | What it does | Platforms | Needs |
|---|---|---|---|
๐ tapAt |
Tap, long press, or double tap at screen coordinates in native, normalized, or percent units | ๐ค ๐ โ | โ |
๐ tapOn |
Tap an element by text/content-desc, resource-id, or Android test tag | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐ค tapAny |
Tap any clickable element | ๐ค ๐ | โ |
๐ swipeOn |
Swipe/scroll on screen or elements | ๐ค ๐ โ | โ |
โ๏ธ swipeAt |
Swipe from one screen point to another, including diagonals, in native, normalized, or percent units | ๐ค ๐ | โ |
๐ซณ dragAndDrop |
Drag and drop element | ๐ค ๐ | โ |
ใฐ๏ธ dragPath |
Drag a finger along a polyline with segment timing, an initial hold and a release hold | ๐ค ๐ | โ |
๐ฑ๏ธ hoverAt |
Hover a stylus or mouse over a path with no contact | ๐ค ๐ | โ |
๐ค pinchOn |
Pinch to zoom | ๐ค ๐ | โ |
๐๏ธ multiTouch |
Simultaneous 2-5 finger tap, long press, or swipe at a screen point | ๐ค ๐ | โ |
๐ค sendKeys |
Execute ordered typing, clear, raw-key, and semantic IME commands | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐น๏ธ focusNavigate |
Move D-pad focus to an element on a focus-driven UI such as Android TV or Apple TV | ๐ค ๐ ๐บ | โ |
โ๏ธ setUIState |
Set multiple form fields by desired state | ๐ค ๐ | ๐ |
๐ selectAllText |
Select all text in focused input | ๐ค ๐ | โ |
๐ pressButton |
Press a device or navigation button | ๐ค ๐ โ ๐บ | โ |
โจ๏ธ keyboard |
Open, close, detect, list/select installed Android IMEs, or switch the AutoMobile typing profile | ๐ค ๐ | โ |
๐ clipboard |
Clipboard operations | ๐ค ๐ | โ |
๐ฃ๏ธ siri |
iOS: activate Siri with a spoken phrase, e.g. to invoke an App Shortcut / App Intent, then return the observation | ๐ค ๐ | โ |
๐ฆ Apps, files & data¶
| Tool | What it does | Platforms | Needs |
|---|---|---|---|
๐๏ธ listApps |
List installed apps on a device, with optional display label and launchable fields when reported by the platform or transport | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐ launchApp |
Launch app by package name | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐ terminateApp |
Terminate app by package name | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐ฅ crashApp |
Intentionally crash a running app through the platform crash path | ๐ค ๐ โ ๐บ | โ |
๐ appLifecycle |
State-preserving background-process kill for saved-state restoration tests | ๐ค ๐ | โ |
๐จ buildApp |
Build an app from source for a simulator or emulator and return the installable artifact path | โ | โ |
๐ฒ installApp |
Install app on device | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐๏ธ uninstallApp |
Uninstall app by package name or bundle identifier | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐ inspectPackageSigning |
Fresh read of one app’s presence and signing | ๐ค ๐ | โ |
๐ getDeepLinks |
Query app deep links | ๐ค ๐ ๐บ | โ |
๐ฅ putAppFile |
Write files into a bounded logical storage target | ๐ค ๐ | โ |
๐ storeKitTest |
Test in-app purchases with StoreKit Testing on an iOS simulator | ๐ค ๐ | โ |
๐งน resetAppLogs |
Reset explicitly named app-container log files on the session’s device, reporting a per-path outcome | ๐ค ๐ | โ |
โ๏ธ getPreference |
Read an Android system property, Android SharedPreferences key, or iOS UserDefaults key | ๐ค ๐ | โ |
๐ง setPreference |
Write an Android system property, Android SharedPreferences key, or iOS UserDefaults key and return read-back verification | ๐ค ๐ | โ |
๐๏ธ deleteKeychainItem |
Delete one exact, host-allowlisted, app-owned iOS generic-password Keychain item | ๐ค ๐ | ๐งฉ |
๐ setKeyValue |
Set app key-value storage entry | ๐ค ๐ | ๐งฉ |
โ removeKeyValue |
Remove app key-value storage entry | ๐ค ๐ | ๐งฉ |
๐งผ clearKeyValueFile |
Clear app key-value storage file | ๐ค ๐ | ๐งฉ |
๐๏ธ listDataStores |
List app Jetpack DataStore instances | ๐ค ๐ | ๐งฉ |
๐๏ธ getDataStore |
Read entries from an app Jetpack DataStore instance | ๐ค ๐ | ๐งฉ |
๐งฎ sqlQuery |
Execute SQL on an app SQLite database, or read a registered Core Data entity on iOS using store and entity | ๐ค ๐ | ๐งฉ |
๐ resetKeychain |
Reset secure test state | ๐ค ๐ | โ |
๐๏ธ setAppLinks |
Android 12+: change an app’s App Links state so https links open the app directly | ๐ค ๐ | โ |
๐ฆ performanceAudit |
Pass/fail performance checks of an app on the current screen, against the configured thresholds | ๐ค ๐ | โ |
โฑ๏ธ measureStartup |
Android: measure cold/warm/hot app startup over N launches | ๐ค ๐ | โ |
๐ getLogs |
Read device logs with filters | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐ป sendBroadcast |
Android: send an explicit broadcast to the app under test with a typed intent, e.g. to toggle a debug flag or seed data | ๐ค ๐ | โ |
๐ ๏ธ backgroundWork |
Android: read and trigger an app’s background work without waiting hours | ๐ค ๐ | โ |
๐ฑ Devices & system¶
| Tool | What it does | Platforms | Needs |
|---|---|---|---|
๐ listDevices |
List booted devices | โ | โ |
๐ฟ listDeviceImages |
List device images | โ | โ |
๐ค getAndroid |
Find or recover an Android AVD and prepare it for automation | โ | โ |
๐ getApple |
Find or recover an iOS Simulator and prepare it for automation | โ | โ |
๐๏ธ provisionDevice |
Provision exact virtual device | โ | โ |
๐งฐ setDeviceResources |
Set selected device resources and verify native state, including disabled iOS service impacts | ๐ค ๐ | โ |
โ๏ธ reconcileDeviceResources |
Compare an iOS Simulator with a requested resource map and report typed drift | ๐ค ๐ | โ |
๐ setActiveDevice |
Set active device using existing session binding or a sessionless per-MCP-connection device pin that lasts for that connection | โ | โ |
๐ killDevice |
Kill device | โ | โ |
โฐ๏ธ deleteDevice |
Stop and permanently delete a device, with verified platform-inventory absence | โ | โ |
๐พ deviceSnapshot |
Capture, restore or delete a device snapshot | ๐ค ๐ | โ |
๐ rotate |
Rotate device orientation | ๐ค ๐ | โ |
๐ portReverse |
Android: manage adb reverse mappings so the app or browser on the device reaches a host server at localhost:<port> |
๐ค ๐ | โ |
๐ setPosture |
Set Android device posture or iPhone Duo simulator posture, with an optional Resizable Android display preset | ๐ค ๐ | โ |
๐ statusBar |
Deterministic status bar for screenshots and video | ๐ค ๐ | โ |
๐ช windowMode |
Android: put an app’s task into fullscreen, picture-in-picture, freeform or split screen with a second app, and read the mode and bounds back | ๐ค ๐ | โ |
๐๏ธ rotaryScroll |
Wear OS: scroll with the rotary crown | ๐ค ๐ | โ |
๐ฅฝ glassesInput |
Android XR glasses touchpad tap and forward/back swipe as focus input | ๐ค ๐ | โ |
๐ ambient |
Wear OS watches only: enter sends KEYCODE_SLEEP, exit sends KEYCODE_WAKEUP, then reads wakefulness and the watch face’s inAmbient from dumpsys wear_service, reported as ambient | ๐ค ๐ | โ |
โ wearSurface |
Wear OS watches only: tiles, watch faces and complications | ๐ค ๐ | โ |
๐ rotateCrown |
watchOS only: rotate the Apple Watch Digital Crown by a signed, non-zero delta in full revolutions, with optional positive velocity | โ | โ |
๐ openLink |
Open URL in browser | ๐ค ๐ ๐บ | โ |
๐ homeScreen |
Go to home screen | ๐ค ๐ ๐บ | โ |
๐ recentApps |
Open recent apps | ๐ค ๐ | โ |
๐ systemTray |
System tray actions for notifications | ๐ค ๐ | โ |
๐ wakeAndUnlock |
Wake a device and unlock its keyguard | ๐ค ๐ | โ |
๐ changeLocalization |
Change locale, time zone, text direction, time format, and calendar system | ๐ค ๐ | โ |
๐ฉบ getDeviceState |
Read device-level state including clock, Do Not Disturb, the connectivity toggles, iOS Simulator biometric enrollment, network condition, and Android emulator virtual sensor values | ๐ค ๐ | โ |
๐๏ธ setDeviceState |
Set device state such as Do Not Disturb, Android connectivity toggles, static location or background route playback on an Android emulator, iOS Simulator, or physical iOS device, iOS Simulator biometric enrollment, and device-wide network condition | ๐ค ๐ | ๐ฉ device-condition |
๐ฅ๏ธ displayConfig |
Read or set the visual display configuration โ font/text scale, effective display density, light/dark theme, and on Android the runtime screen size, navigation mode and display-cutout emulation โ for adaptive-layout, edge-to-edge and large-font accessibility testing | ๐ค ๐ | โ |
๐จ displayProfile |
Android: apply an OEM-style display profile in one call, composing displayConfig and setPosture |
๐ค ๐ | โ |
๐ฎ getSessionRequirementCapability |
Report the OS a session device requirement would resolve to on this broker host and whether it is satisfiable now, without acquiring a device | โ | โ |
๐ชช biometricAuth |
Simulate biometric auth: Android emulator/SDK hook, or iOS via the app’s AutoMobile SDK or Simulator | ๐ค ๐ | โ |
๐ณ shake |
Shake device; iOS Simulator and Android emulator only | ๐ค ๐ | โ |
๐งฒ setSensor |
Set or reset one Android emulator virtual sensor through the emulator console | ๐ค ๐ | โ |
๐๏ธ injectAudioInput |
Play a WAV file or a built-in tone into an Android emulator’s or iOS Simulator’s microphone, so an app under test records known audio | ๐ค ๐ | โ |
โน๏ธ stopAudioInput |
Stop an audio injection started by injectAudioInput, on the device it runs on |
๐ค ๐ | โ |
๐ captureAudio |
Record up to 20 s of an Android emulator’s or iOS Simulator’s audio output and report its level | ๐ค ๐ | โ |
๐ phoneCall |
Simulate a phone call: Android emulator gsm commands, or CallKit via the app’s AutoMobile iOS SDK | ๐ค ๐ | โ |
๐ฌ sendSms |
Send simulated incoming SMS: Android emulator, or a notification via the app’s AutoMobile iOS SDK | ๐ค ๐ | โ |
๐ฑ seedContent |
Seed Android contacts, calendar events, call history and SMS inbox, tagged automobile.<namespace> |
๐ค ๐ | โ |
๐ต mediaSession |
Android media sessions: read playback state, position, speed, metadata and actions | ๐ค ๐ | โ |
๐ฃ postNotification |
Post notification via Android SDK hooks or iOS/watchOS/tvOS simulator simctl push | ๐ค ๐ โ ๐บ | โ |
๐ getNotificationPolicy |
Read app notification/DND policy state | ๐ค ๐ | โ |
๐คซ setNotificationPolicy |
Set app notification/DND policy state | ๐ค ๐ | โ |
๐ getAppPermissions |
Read app permission state | ๐ค ๐ | โ |
๐ซ setAppPermissions |
Grant or revoke app permissions | ๐ค ๐ ๐บ | โ |
โ ๏ธ killProfile |
Android: simulate an OEM background-kill policy on a stock device with standby buckets, appops, Doze, permission revocation, foreground-change kills, timed screen-off cleanup, boot-receiver observations and force-stop | ๐ค ๐ | โ |
๐ญ manufacturerProfile |
Android: apply an OEM manufacturer profile on a stock device by composing the keyboard profile, killProfile and displayProfile executors |
๐ค ๐ | โ |
โ ๏ธ systemAlerts |
Set how OS permission and system alerts are handled before and after each UI interaction on this device, including executePlan steps |
๐ค ๐ | โ |
๐ Network, plans & recording¶
| Tool | What it does | Platforms | Needs |
|---|---|---|---|
๐ถ network |
Control network capture and error simulation | ๐ค ๐ | ๐งฉ |
๐ญ mockNetwork |
Add mock network response rule | ๐ค ๐ | ๐งฉ |
๐งฝ clearMockNetwork |
Clear mock network response rules | ๐ค ๐ | ๐งฉ |
๐ getNetworkGraph |
Get aggregate captured network graph | ๐ค ๐ | ๐งฉ |
โถ๏ธ executePlan |
Execute YAML plan steps | ๐ค ๐ โ ๐บ ๐ฅฝ | โ |
๐ criticalSection |
Synchronize multiple devices at a barrier, then run steps serially | ๐ค ๐ | โ |
๐ง barrier |
Synchronize multiple devices at a barrier, then let all proceed concurrently | ๐ค ๐ | โ |
โบ๏ธ recordToolCalls |
Record MCP tool calls to a YAML plan | โ | ๐ฉ tool-call-recording |
๐ฌ recordDeviceGestures |
Start recording on-device user input for exportPlan |
๐ค ๐ | โ |
๐ค exportPlan |
Stop the active recordDeviceGestures or recordToolCalls recording and export a YAML plan |
โ | โ |
๐ฅ videoRecording |
Start or stop device video recording | ๐ค ๐ ๐บ | โ |
๐ performanceTrace |
Record a native performance trace | ๐ค ๐ | โ |
๐ฌ captureTrace |
Android: record a Perfetto trace from typed categories | ๐ค ๐ | โ |
๐ง heapDump |
Dump a running app’s heap to a host file | ๐ค ๐ | โ |
โฟ Accessibility & tooling¶
| Tool | What it does | Platforms | Needs |
|---|---|---|---|
๐ฆฎ accessibility |
Check or control accessibility services | ๐ค ๐ | โ |
๐ฒ accessibilityFocus |
Set accessibility focus on Android and iOS by resourceId, text, or contentDesc | ๐ค ๐ | ๐ |
๐ซถ accessibilityAction |
Perform an Android accessibility action on an element, the way TalkBack or Switch Access does | ๐ค ๐ | โ |
โ
accessibilityAudit |
Audit the accessibility of the current screen | ๐ค ๐ | โ |
๐งฉ setToolEnabled |
Control tools/list discovery for this MCP session | โ | โ |
๐บ TV focus navigation¶
focusNavigate({ until, direction?, maxSteps?, settleMs?, select? }) (#14947) drives focus-based UIs on Android TV and
Apple TV. It observes, and while the input-focused element (focusedElement) does not match until (text,
contentDesc, resourceId, or a lone label/elementId resolved to the element tapOn would tap) it presses the
D-pad arrow toward the nearest match, or direction when given, then observes again. With no focused element or no
match on screen it presses down. When a press leaves focus unchanged, or keeps moving focus without getting closer,
the other arrows are tried. It returns the path of focused elements and a status: focused, max_steps
(default 30, at most 100) or stuck (every arrow from the focused element was tried; an explicit direction stops
after two unchanged presses). With select: true it presses enter once focused and reports selected. On Apple TV
the arrows are Siri Remote directional presses, select presses the remote’s select button, and focus is re-read
after each press (see docs/how-it-works/plat/android/form-factors.md and
docs/how-it-works/plat/tvos/xcuitest-spike.md).
๐ Notes¶
- โจ๏ธ Compact observations fold soft-keyboard keys into one
keyboard: { visible: true, package: "โฆ" }summary plus at most one skeleton row (<ime> | Keyboard (โฆ) | input). The row appears only with at least one bounded accessible descendant. - ๐ธ Screenshot paths are kept for at least 10 minutes after return unless evicted early under capacity pressure; each path has a
screenshotExpiresAt/expiresAt(host epoch ms, not a promise). Retention starts at 128 MiB and scales by 64 MiB per active device up to 1 GiB, within a limit of 4096 files. Each deviceโs newest 3 screenshots (counting its incoming frame) are evicted only as a last resort, and a capture is admitted even if nothing fits. The byte limits and per-device allowance are configurable. After a restart, existing files get a process-start grace of ten minutes. - ๐งน
holdSessionOnFailurekeeps plan sessions instead of auto-releasing them.
No audio input (audio_input_unavailable)¶
When a device cannot record audio, observe returns diagnostics: [{ code: "audio_input_unavailable", message, retryable: false }],
and a setAppPermissions grant of microphone (iOS) or RECORD_AUDIO (Android) still succeeds but carries the same
diagnostics entry. This applies to an iOS simulator on a host with no audio input device, and to an Android emulator launched
with -no-audio (AUTOMOBILE_EMULATOR_AUDIO=false) or with hw.audioInput=no. Recordings on that device contain no audio.
Treat this as an environment limitation: do not retry, and do not report it as an app failure. Fix the host (for example, make a
virtual loopback device the default input) or the AVD audio settings. The message names the cause.
Stylus pointer on tapAt, swipeAt and dragPath¶
pointer: { toolType: "stylus" } injects a stylus instead of a finger (Android only; never falls back to a finger).
pointer shape |
Route | Needs |
|---|---|---|
{ toolType: "stylus" }, pressure omitted or 1 (tapAt, swipeAt) |
adb shell input stylus |
Android 12+ (API 31) |
pressure other than 1, tilt (radians, 0 to PI/2), orientation (radians), or buttons (primary, secondary) |
CtrlProxy instrumentation | stylus_input_v1 (Android 12+) |
dragPath with any pointer |
CtrlProxy instrumentation | stylus_input_v1 (Android 12+) |
On an emulator or userdebug build whose adb shell already runs as root (after adb root) and has the uinput tool, a stylus
stroke with pressure, tilt, orientation, buttons and a dragPath also works without stylus_input_v1, through a
virtual pen; each stroke starts about 4 seconds later while the pen attaches.
pointer: { toolType: "eraser" } touches with the pen’s eraser end (apps see TOOL_TYPE_ERASER) on tapAt, swipeAt and
dragPath, with optional pressure, tilt, orientation and buttons. It needs that same root adb shell with the
uinput tool, and is refused before any input everywhere else.
dragPath points also take pressure and tilt (they need pointer), interpolated between points; holdMs and
releaseHoldMs become holds at the first and last point. A device or runner that cannot honour the request is refused before
anything is sent, naming the missing capability. tilt, orientation and buttons are Android only; doubleTap and a panel
display are not supported with a stylus yet. Details and the unverified parts are in the
stylus design note.
hoverAt¶
Hover enter, move and exit over points with no contact (Android only; the reasons iOS is refused are in the
stylus design note). It takes the dragPath point shape
(x, y, per-point durationMs) plus an optional per-point distance (stylus only), and the same
coordinateSpace, display, snapshotId and layer.
toolType chooses who hovers, and apps see the difference:
toolType |
What the app receives |
|---|---|
stylus (default) |
ACTION_HOVER_* with TOOL_TYPE_STYLUS, SOURCE_STYLUS and an AXIS_DISTANCE value |
mouse |
ACTION_HOVER_* with TOOL_TYPE_MOUSE and SOURCE_MOUSE; no distance; desktop-style UI |
It needs the hover_input_v1 runner capability, or, for a stylus hover, an adb shell that already runs as
root with a writable /dev/uinput (after adb root on an emulator or userdebug build). A device or runner without either, and iOS, are refused before
anything is sent with the missing capability named, and a hover is never turned into a touch. CtrlProxy
advertises the capability and injects the hover through its UiAutomation instrumentation (see the
stylus design note). The tool type, source
and hover actions above are confirmed on API 31 to 37 emulators; the stylus AXIS_DISTANCE value and physical pen
devices are unverified.
prototype¶
Show agent-authored UI on a device: show, patch, inspect, dismiss {id} / {all: true}, status. Android can hold up to four prototypes (replaceOthers: false); iOS shows one at a time. Features (data model, gestures, checks) require the matching device capability and are refused before upload otherwise. Spec: prototype UX. Camera and media support per runtime: camera and media guide.
Watch-sized mockups¶
Prototypes cannot run on watchOS, but a watch-sized spec gives a quick visual mockup of a watch
screen on an iPhone simulator or Android emulator. Float the window in the centre and give the root
box the watch screen’s size, a large corner radius (which also clips its content) and a black
background:
| Case | width ร height (dp/pt) |
cornerRadius |
|---|---|---|
| 41 mm | 176 ร 215 | 36 |
| 45 mm | 198 ร 242 | 40 |
| 49 mm Ultra | 205 ร 251 | 44 |
{
"id": "watch-41mm",
"window": {
"placement": { "type": "floating", "gravity": "center", "offset": { "x": 0, "y": 0 } }
},
"components": {
"watchRow": {
"root": {
"type": "row",
"style": {
"width": "fill",
"background": "#222222",
"cornerRadius": 16,
"padding": { "start": 10, "end": 10, "top": 8, "bottom": 8 }
},
"children": [{ "type": "text", "text": "{props.title}", "style": { "color": "#FFFFFF" } }]
}
}
},
"root": {
"type": "box",
"style": {
"width": { "dp": 176 },
"height": { "dp": 215 },
"cornerRadius": 36,
"background": "#000000"
},
"children": [
{
"type": "scroll",
"child": {
"type": "column",
"style": { "spacing": 6, "padding": { "start": 8, "end": 8, "top": 12 } },
"children": [
{ "type": "use", "component": "watchRow", "props": { "title": "Drink water" } },
{ "type": "use", "component": "watchRow", "props": { "title": "Stand up" } }
]
}
}
]
}
}
Put each common watch pattern in components and reuse it with use: a vertically scrolling list
of rows, a full-screen metric (one large number with a label and unit), a two-button confirmation
(two buttons with weight: 1 in a row, swapped in with visibleWhen), and a now-playing card
(title, artist, previous / play-pause / next iconButtons and a linear progress).
The mockup is a phone overlay sized like a watch. It does not simulate the Digital Crown, Always-On or wrist-down dimming, real watchOS or Wear OS fonts and controls, or the round Wear OS screen, and nothing runs on a watch. Check the final design on a real watch. To drive a watch app itself, see Apple Watch simulators.