Apple Watch tool behavior¶
How each tool behaves on an Apple Watch simulator. Every call names the watch
with platform: "watchos".
observe¶
observe returns a screenshot. With the automation runner connected, it also returns the target
app’s view hierarchy and on-screen system alerts, even when the app is not in the foreground. Those
alerts include the notification permission prompt and the notification short look and long look
with action buttons.
- If AutoMobile cannot read the hierarchy, the result carries
hierarchyUnavailableand no hierarchy fields. - When the target app is not in the foreground, the message says to call
launchApp, or to post a notification and retryobserve. Read the screenshot meanwhile. - When the app
launchAppstarted has exited or crashed,hierarchyUnavailablecarriesappNotRunning: { bundleId }. ReadgetLogsfor the crash and calllaunchAppto start it again. - With a hierarchy, bounds and
screenSizeare in points andscreenshotImageSizeis in pixels. A screenshot-only result reportsscreenSizein pixels. CheckscreenSize.units(pointsorphysical-pixels) before using it fortapAt, which takes points. - With the runner connected, an element
waitForre-reads the hierarchy until it matches ortimeoutelapses.
The watch refuses these options: waitFor with for, posture, activeDisplay or
activeWindow; waitFor without a runner; settled, raw, includeExtras, scope, layer,
skipBackStack, crop, screenshotOptions and screenshot: "none".
tapOn and tapAt¶
tapOn takes a selector with an exact elementId or text, and an optional index. It supports
only action: "tap". Only elements wholly on screen match, including notification action buttons.
The notification permission prompt scrolls: use rotateCrown to bring Allow wholly into view
before you select it.
tapAt takes absolute coordinates in points from observe, within screenSize. A point outside
the watch screen fails with tap is outside the watch screen.
Outside the Smart Stack, both results include effect: {screenChanged, basis}. AutoMobile checks
the screen for about 1 second after the tap. basis is viewHierarchy changed,
viewHierarchy unchanged or insufficient observation data. The tap is never retried for you. A
changed screen does not prove the intended control responded, so observe when it matters.
When a system or app alert, such as the notification permission prompt, covers the app, tapOn
refuses a target outside the alert. The error names the alert title and its buttons. Tap one of
those buttons first, for example tapOn with text: "Allow", then retry the original tap.
tapAt has no such guard: a point under an alert taps the alert.
With the host app not in the foreground and no Smart Stack open, tapAt taps the watch face at the
given point. Whether such a tap opens a complication is unverified, so observe afterwards. In that state tapOn still needs the host app in the foreground.
Smart Stack widgets¶
When the Smart Stack is open and the host app is not in the foreground:
tapOnmatches a widget’s exact text or cell identifier on the watch face. Only cards wholly on screen match. AutoMobile refuses text that only an off-screen widget shows, with a hint to scroll the Smart Stack first.tapAttaps widgets by point coordinates too.observereturnshierarchyUnavailableand no hierarchy fields.
Tapping a widget opens its host app, where the normal tools work. A widget you just added shows its text about 2 seconds after the final Done tap; wait that long before you match it. See Smart Stack for how this works.
swipeOn and pressButton¶
swipeOn swipes in a direction, including on the watch face after pressButton with home. With
lookFor, it swipes until an exact text or elementId is on screen, up to maxSwipes (default
10, maximum 100). Each lookFor step drags slowly by about one row, so reads overlap and
skip no nearby targets. You cannot set coordinates or duration.
pressButton supports home only.
rotateCrown¶
rotateCrown turns the Digital Crown. Enable it with setToolEnabled (see
Dynamic Tools).
{ "name": "rotateCrown", "arguments": { "platform": "watchos", "delta": 2 } }
| Parameter | Meaning |
|---|---|
delta |
Signed, non-zero, in full crown revolutions: 1 is one complete turn, 0.25 a quarter turn. The sign sets the direction |
velocity |
Optional, positive. Default 1 |
The app’s crown binding (step size and sensitivity) decides how far a turn moves a control. One
revolution can cross a whole small control. On a 0–10 control with step 1 and medium sensitivity,
0.25 moves about 3 steps, 0.5 about 7, and 1 reaches the end. To reach an intermediate value,
use fractional deltas and observe between turns. A turn smaller than one step (for example 0.1
on that control) may not register.
How success is judged¶
The focused control is the innermost focused element with an accessible value (for example SwiftUI
.accessibilityValue), or the one valued element inside a focused view that does not scroll. When
the app reports no focused element, as with SwiftUI digitalCrownRotation, it is the app’s one
valued control outside scrolling content.
- With a readable focused value before and after, that value must change for success.
- Otherwise any hierarchy change counts as success. An unrelated update, such as a status label, can satisfy it, so success does not confirm that the crown moved the control.
- Without a readable focused value, the tool first touches an interior point clear of elements to wake crown routing. Content absent from the hierarchy, such as a canvas that handles taps, can still receive that touch. To avoid it, tap the control that should respond first.
- If nothing changes, it may wake again and retry once. That needs a stable screen and an unchanged readable focused value. After an initial wake it never repeats the turn without a comparable value.
- Without a stable hierarchy, it compares the last pre-rotation value with the post-rotation value, without waking or retrying. If either is unavailable, it skips verification.
When a rotation changes nothing, the error says which case applies:
- The focused value stayed the same: the error quotes it (
value stayed "10"). The turn was too small, or the control is at its limit. It is at its limit only when the value already sits at the end the turn heads toward, such as"10"after a positivedeltaon a 0–10 control. Otherwise, including"0"after a positivedelta, retry with a largerdeltain the same direction. - No focused control exposes a readable value: no view may hold crown focus, or the scrolled
content is at its limit. Tap the control first. For a
deltabetween -1 and 1, the error also says the turn may be too small.
Call observe after rotating. Retry only if it shows the turn had no effect, since a retry after a
working turn can overshoot, and tap the control first.
sendKeys¶
sendKeys accepts non-empty text only and appends to the focused field. Focus a field with tapOn
first, or pass a selector with an exact elementId or text. Selector text matches a field’s
label, value or placeholder, and prefers text fields over static text. Like tapOn, a selector
refuses a field covered by an alert and names its buttons; tap one, then retry. A focused field opens the
watch keyboard, and AutoMobile types through it. Input fails when no text field or keyboard is up.
The watch refuses commands that clear, replace or press keys.
postNotification¶
postNotification sends a simulated push to the watch app named by appId. The simulator cannot
pre-grant notification authorization, so the app must request permission. Find Allow with
observe and tap it with tapOn, scrolling with rotateCrown or swipeOn first if needed. A “CtrlProxyWatch” permission prompt may appear first.
- If the watch dropped the push because the app’s notifications are denied,
postNotificationreturnssuccess: false. That happens after Don’t Allow, or when Allow was never reached. - The watch does not ask a denied app again. Uninstall and reinstall it (
installAppresets its permissions), launch it, answer Allow, then post again. - When no delivery verdict is available, success means the simulator accepted the push.
- A banner disappears after a few seconds. Call
observeright after the push to inspect its short look; the long look exposes its action buttons.
App management¶
installApptakes a watch simulator.appbundle.uninstallApprefuseskeepData: true.launchAppreports an app that is not installed without launching anything. It refusesprototype: true. WithclearAppData, it wipes the app’s Documents, Library and tmp before launch. It does not observe the launched app, soverifiedstays unset withverificationSkipped: "platform".terminateAppsucceeds withwasRunning: falsewhen the app was not running.crashAppsends SIGABRT to the running watch app named byappId, as on an iOS simulator. It checks the watch’s unified log and the host crash reports for fresh crash evidence. The result hasconfirmed,evidenceand, when it finds the report, acrashReportsummary. An app that is not running fails withwasRunning: false, and AutoMobile signals nothing.
To refresh a widget on demand, make the host app reload its widget timelines when it sees a launch
argument you choose. Then call terminateApp and launchApp with that argument, for example
launchArguments: ["-reload"]. The widget shows the new content within about 2 seconds, with no
tap in the host app.
getLogs¶
getLogs reads the watch simulator’s unified log as on an iOS simulator. Enable it with
setToolEnabled (see Dynamic Tools).
| Parameter | Watch behavior |
|---|---|
appId |
Required. Returns lines from the app’s subsystem or its process |
minPriority |
V/D debug and up, I info and up, W default and up (the unified log has no warning level), E error and fault, F fault only |
tags, buffers, includeSystem |
Android-only; ignored with a warning |
executePlan¶
executePlan replays a plan on a watch with platform: "watchos", including a plan that
recordToolCalls recorded on a watch. Each step runs as the same tool called directly. A step whose
tool does not accept a watch fails with the standard refusal. Plans on a watch record no video.