Golden capture and replay for the iOS CtrlProxy¶
Unit tests can replay real runner traffic at two layers:
| Layer | What is recorded | Switch | Replayed by |
|---|---|---|---|
| Host WebSocket | Every request the TS IOSCtrlProxyClient sends and every runner reply |
AUTOMOBILE_IOS_CTRLPROXY_RECORD_DIR on the host process |
test/fakes/ReplayCtrlProxyWebSocket.ts |
| Runner merger inputs | Each (xcuitest, sdk) pair HierarchyMerger receives on request_hierarchy |
CTRL_PROXY_IOS_HIERARCHY_PAIR_DIR in the runner’s environment |
ios/control-proxy/Tests/CtrlProxyTests/HierarchyMergerGoldenReplayTests.swift |
Recording format¶
Host exchanges. CtrlProxyExchangeRecorder wraps the client’s WebSocket
factory. It writes one pretty-printed JSON file per exchange, numbered in arrival
order:
NNNN-<request type>.jsonholds{"request": …, "response": …}, paired byrequestId.NNNN-<request type>.jsonholds{"request": …}for a frame that gets no reply.NNNN-push-<type>.jsonholds{"push": …}for an unsolicited message, such asconnected,hierarchy_updateorperformance_update.
Per owner policy, nothing is redacted except text typed into a password field. The
host never sees which field it types into. So for request_set_text and
request_append_text, the recorder looks up the target in the latest recorded
hierarchy (by resourceId, or else the focused node). It replaces text with
<redacted:password-field> when that node has password: "true" or the target
cannot be found.
Merger pairs. HierarchyPairFileRecorder writes compact, sorted-key
{"sdk": SdkViewHierarchy, "xcuitest": ViewHierarchy} files named
pair-NNNN-<bundle>.json. It records only when the runner has an SDK hierarchy for
the foreground app. Simulators share the host filesystem, so the files land
directly in the named host directory. Physical devices are not supported, and the
daemon forwards the variable only to simulator runners. XCUITest already masks
secure-field values, and the SDK tree carries labels and geometry rather than field
contents, so nothing is redacted.
Capturing or refreshing¶
Use a simulator you create for the purpose. The committed corpus came from an iPhone 17 on iOS 26.5.
- Build the runner from your checkout:
AUTOMOBILE_CTRL_PROXY_IOS_DERIVED_DATA=/tmp/golden/dd bash scripts/ios/ctrl-proxy-build-for-testing.sh. - Copy the generated
.xctestrunand add these keys underCtrlProxyUITests.EnvironmentVariables:CTRL_PROXY_IOS_PORT(for example8791),AUTOMOBILE_DEVICE_ID(the simulator UDID) andCTRL_PROXY_IOS_HIERARCHY_PAIR_DIR(the pair output directory). Then start the runner:xcodebuild test-without-building -xctestrun <copy> -destination "platform=iOS Simulator,id=<udid>" -only-testing:CtrlProxyUITests/CtrlProxyUITests/testRunService. - Drive it with
scripts/ios/capture-ctrlproxy-golden.ts, which talks to the runner directly with no daemon:# Host exchanges: launch Settings, observe, tap "General", observe. AUTOMOBILE_IOS_CTRLPROXY_RECORD_DIR=/tmp/golden/settings \ bun scripts/ios/capture-ctrlproxy-golden.ts --udid <udid> --port 8791 \ --bundle com.apple.Preferences --tap General # Merger pairs: one fresh pair per Playground tab, and per demo screen. SIMCTL_CHILD_PLAYGROUND_INITIAL_TAB=demos AUTOMOBILE_IOS_CTRLPROXY_RECORD_DIR=/tmp/golden/x \ bun scripts/ios/capture-ctrlproxy-golden.ts --udid <udid> --port 8791 \ --bundle dev.jasonpearson.automobile.Playground --fresh --tap "Scroll Performance" - Copy the results to
test/fixtures/ios/ctrlproxy-golden/<scenario>/(host exchanges, unmodified) ortest/fixtures/ios/merger-pairs/<name>.json(pairs). Re-run both replay suites, then update the expected values inHierarchyMergerGoldenReplayTestsand the results below. Every fixture directory is pinnedtext eol=lfin.gitattributes.
With a daemon, set AUTOMOBILE_IOS_CTRLPROXY_RECORD_DIR and
CTRL_PROXY_IOS_HIERARCHY_PAIR_DIR on the daemon process. The client then records
everything that daemon’s tools send, and the daemon forwards the pair directory
into the simulator runner’s xctestrun environment. Isolate a private daemon fully
first; see docs/using/environment-variables.md.
Committed corpus¶
test/fixtures/ios/ctrlproxy-golden/settings-tap-general/holds 14 exchanges. They arerequest_launch_app, twoget_sdk_capabilities,request_hierarchy_if_staleon the Settings root,request_tap_coordinatesat the General row’s centre (201, 406),request_hierarchy_if_staleon the General screen, and the pushes in between.test/features/observe/ios/IOSCtrlProxyClientGoldenReplay.test.tsreplays them through a realIOSCtrlProxyClient.test/fixtures/ios/merger-pairs/holds seven Playground pairs: the four tabs (--freshlaunch into each) and the SDK Status, Scroll Performance and Animations demo screens (--fresh --tap). The merger only consumes the SDK tree’s wire shape, so the Playground build date does not affect the result.
Nearest-match tie-break results¶
HierarchyMerger.ToleranceTieBreak.legacyDeltaLoopOrder reproduces the
pre-#8662 probe. Exact bounds are tried first. Then the first (dl, dt, dr, db) in
ascending nested-loop order wins, where each key is represented by its
first-inserted node, and a representative that fails the identifier check makes
that key a miss. The replay merges every pair both ways and compares every output
node’s own fields.
| Fixture | XCUITest nodes with bounds | Exact SDK bounds | Within ±2 only | …at ≥2 distinct bounds | Output differences |
|---|---|---|---|---|---|
| tab-discover | 46 | 23 | 10 | 0 | 0 |
| tab-demos | 69 | 31 | 14 | 0 | 0 |
| tab-files | 14 | 7 | 3 | 0 | 0 |
| tab-settings | 52 | 32 | 11 | 0 | 0 |
| demo-sdk-status | 45 | 28 | 5 | 0 | 0 |
| demo-scroll-performance | 46 | 21 | 3 | 1 | 0 |
| demo-animations | 20 | 10 | 4 | 0 | 0 |
| Total | 292 | 152 | 50 | 1 | 0 |
The two orders can disagree only for a query with SDK candidates at two or more distinct bounds within
tolerance; same-bounds candidates resolve to the first node in document order either way. Synthetic
layouts where differently sized views sit within 2 pt of an element are covered by
HierarchyMergerToleranceMatchTests.
testCorpusToleranceCoverageIsAsDocumented pins the coverage columns, so an empty
diff cannot pass vacuously. Making the legacy matcher always miss turns
testRealPairsDifferOnlyWhereDocumented red.
Known differences between replay and a live runner¶
- Matching is by type and order, not by payload. A replayed request gets the
next unused recorded response of the same
type, with the liverequestIdsubstituted. Coordinates, timeouts and frame contexts are not compared; tests assert onsentRequestswhen they matter. A request with no recorded response left goes intounmatchedRequestsand gets no reply. - Pushes are not replayed, except
connected. The replay socket replays the handshake when the socket opens. The fixture keeps recordedhierarchy_updateandperformance_updatepushes, but the replay socket never emits them on its own. - Recorded timing.
perfTiming,timestampandupdatedAtare capture-time values, and replies arrive on the next microtask with no latency. - Redaction. Replayed text-input requests carry
<redacted:password-field>wherever the recorder redacted them. - The runner’s SDK cache. On
request_hierarchy, the runner probes the SDK only while it has no cached tree for the foreground app. After that it reuses the cache until the app posts/sdk-events, so a pair recorded right after an in-app tap can pair the new XCUITest screen with the previous screen’s SDK tree. The--freshflag clears the cache. It requests a SpringBoard hierarchy before each capture, after terminating the app or (after a tap) going Home, then relaunches. Every committed pair has an SDK timestamp later than the screen change. - Switching apps outside the runner. After
simctl launchalone, the runner reported the new app’s hierarchy, but taps still targeted the previous app and failed withApplication … is not running. The capture script therefore follows everysimctl launchwithrequest_launch_app, which activates the app and retargets the runner. The daemon’slaunchApptool already takes this path.