Stylus, pressure and hover injection spike¶
- Kind: decision
- Date: 2026-10-19
- Source: moved from
how-it-works/mcp/stylus-pressure-hover-injection.md; #12539, #6335, #13082, #13096, #13097, #13098, #13099, #13101, #13103, #13105, #16028, #13093 - Shaped: Stylus, pressure and hover injection
Context¶
Research spike #12539: what each platform can inject beyond a plain finger touch (tool type, pressure, tilt, stylus buttons, hover, eraser), and how AutoMobile should expose and record it. At the start of the spike:
- CtrlProxy built
GestureDescription.StrokeDescription(path, start, duration[, willContinue])and calleddispatchGesture; only aPathand timing went in.dispatchGestureevents were reported to arrive asTOOL_TYPE_UNKNOWN, which some apps (DocumentsUI) ignore (#6335). - The adb fallback (
src/features/action/touchscreenInput.ts) hard-wired thetouchscreensource. GestureOptions.pressure(src/models/GestureOptions.ts) had no readers.- The iOS runner’s
ObjCExceptionCatcher.hdeclared seven private selectors, none for pressure, hover or Pencil. Non-touch IOHID injection had precedent (keyboard events inGesturePerformer+HardwareButtons.swift; the hinge-angle event inDefaultHingeAngleSetter.swift). - Candidate Android routes also included hidden
InputManager.injectInputEvent(INJECT_EVENTSsignature permission, reachable viaapp_processas shell uid, greylisted on recent releases). Another wassendeventinto an existing/dev/input/eventNnode (root, slow per-line events, useful only if the digitizer already advertises pen axes).
Options¶
API shape:
- Optional pointer fields on the existing coordinate tools (
tapAt,swipeAt,dragPath) plus a smallhoverAttool. - New tools per input kind (not chosen).
Android routes behind one interface, chosen by required features: ctrlproxy-a11y,
adb-input-source, instrumentation, uinput, emulator gRPC.
Decision¶
- Same gesture, different pointer. Pointer fields go on the coordinate tools rather than new
tools: tool count and schema size already burden
tool-definitions.json, and defaults keep current calls unchanged. A stylus has no accessibility-element meaning, so element tools do not take it.hoverAtis its own tool because hover has no contact. - Capability flags, never a silent downgrade. Following the CtrlProxy
*_v1convention (gesture_display_id_v1,multi_finger_gesture_v1), the runner advertisesstylus_input_v1/hover_input_v1; a request needing one is refused before dispatch on older APKs. A host-sidepointerInputsummary is derived from the flags and chosen route, never a second source of truth. Tilt and buttons stay Android-only until iOS can honour them. - iOS simulator (#13103): no-go for pressure, tool type, tilt, eraser and buttons. Hover was the
only go: a mouse pointer path (public
hover()first, private mouse selectors as fallback) behind a runnerhover_input_v1flag absent when neither resolves. - Instrumentation route (#13097): GO. On an API 37
google_apisuserdebug AVD the Pointer Probe (#13096) received strokes asTOOL_TYPE_STYLUS, sourceSTYLUS(0x4002), with planned pressure and tilt, orientation, andbuttons: 32asSTYLUS_PRIMARY. Non-root hover arrived as STYLUS, or MOUSE with tool"mouse". - uinput route (#13105): PASS on Android 17 / API 37 userdebug:
SOURCE_STYLUSwith X/Y, normalized PRESSURE 0..1, ORIENTATION, DISTANCE and TILT 0..1.571 radians, without idc/keylayout files. The probe received pressure 0.98 from raw 4000/4095, tilt 0.55 andSTYLUS_PRIMARY(32). A productionuinputroute was conditional on a need for rooted-device or userdebug-emulator record/replay symmetry; instrumentation is the replay route.
Consequences¶
- Route selection (#13098), the uinput route (#13105), the instrumentation route (#13097), rich
stylus on
tapAt/swipeAt/dragPath(#13099) and recorder pen parsing (#13101) followed. - Instrumentation latency on a loaded API 37 AVD host: a one-point hover took 132-162 ms from
am instrumentstart to exit, injection itself 3-10 ms. A 1 s, 65-event stroke ran in 1.1-2.1 s. These are not a budget. - Planned recorder work at the time: surface
distance, a sampledpressurearray for strokes, and newhoverandstrokeevent types, with replay failing loudly if a route cannot honour them. - Open experiments at the time:
- a physical pen device;
- a rooted physical device’s
uinputpen; - emulator gRPC pen and console
event sendagainst the probe; - on a physical iPad,
pressDownWithPressure:atOffset:and rawXCPointerEvent.pressureagainstUITouch.force. An IOHID digitizer event through thedlsympattern was not tried; Pencil also needs entitlement checks. -
16028 stayed open for device acceptance of the stuck-pointer host protection.¶