macOS CI capacity plan (#11010, #11011, #11012)¶
Owner steps below are still open. The sourced CircleCI plan numbers are in the earlier CircleCI migration plan (#10887).
Rule: every pull-request macOS job runs in exactly one place. Hosted
GitHub xcode-27 (macos-26 for jobs not on the Xcode pin) is the fallback for
fork PRs and, per run, for a self-hosted pool with no online runner.
Pools¶
| Pool | Concurrency | Carries |
|---|---|---|
CircleCI m4pro.medium |
1 macOS job at a time | PR Playground tests, all post-merge and nightly macOS work |
Self-hosted Mac automobile-mac |
N runner processes (N = 4 recommended) | Small, non-simulator PR jobs |
Self-hosted Mac automobile-mac-heavy |
exactly 1 runner process | PR xcodebuild and simulator jobs, one at a time |
Namespace auto-mobile-macos |
1 macOS job (trial) | iOS Device Capture to WHEP |
CircleCI (#11010)¶
CircleCI runs one macOS job at a time for this organization, so it carries one PR job plus the work off the PR critical path.
Pull requests¶
ios-playground-tests is the only CircleCI job on the PR path (gated on
run-ios, which path filtering sets from the same paths as filter-ios in
pull_request.yml). It is advisory and must stay off the required-check list.
Swift Packages, Build Xcode Projects and Build Root SPM Package run on GitHub
only; their CircleCI mirrors are gone.
Post-merge¶
merge.yml schedules no macOS job. On every push to main the setup pipeline’s
detect-main-macos-changes job diffs HEAD~1..HEAD against
.circleci/main-macos-paths.txt:
| Path group | Paths | CircleCI job |
|---|---|---|
run-main-ios |
ios/**, scripts/ios/**, Package.swift, SwiftFormat/SwiftLint config |
Build Xcode Projects (main): generate, drift check, build |
run-main-desktop |
android/desktop-app/**, android/desktop-core/** |
Build Desktop App (macos) |
Both groups also match .circleci/**, so a config change revalidates itself.
Apart from the small setup jobs in config.yml, all Linux CI runs on GitHub
Actions.
Swift coverage keeps its trigger: merge.yml’s swift-code-coverage job
starts a run-swift-coverage-main pipeline for coverage-input changes and
downloads the badge.
Coalescing. Each post-merge job first runs
scripts/ci/circleci-main-superseded.sh <group>. If a newer main commit changes a
path in the same group, that commit’s pipeline runs the job on a tree containing
this one, so the older job halts (green) before doing any work. The last
commit in a burst that touches the group is never superseded, so a burst of N
merges costs one run for that commit. The check fails open: if it cannot read
main, the job runs. CircleCI’s project-level “auto-cancel redundant workflows” is
not used: it ignores the default branch and would cancel an iOS run in favour of
a newer pipeline with no iOS changes.
Nightly¶
nightly.yml schedules no hosted macOS job. A CircleCI scheduled pipeline on main
with run-nightly-macos=true runs the nightly-macos workflow, chained in this
order with terminal requirements so a failure never skips later jobs:
- XCTestRunner Simulator Tests (the PR
run-ios-simlabel and contract-input runs still use.github/workflows/xctestrunner-simulator-tests.yml; a policy test keeps the selected tests and daemon budget identical) - Swift Packages Sweep (Xcode 26.6)
- Build Xcode Projects Sweep (Xcode 26.6)
- macOS Node Unit Tests
- macOS Node Host Integration Tests
- macOS BATS Shell Tests
- macOS BATS Integration Tests
- XCTestRunner Thread Sanitizer
CircleCI posts no commit status for scheduled pipelines (#11615), so
.github/workflows/circleci-nightly-visibility.yml (ubuntu, daily 10:40 UTC, not a
required check) runs scripts/ci/report-circleci-nightly.sh. It reads the nightly
through the CircleCI API with the existing CIRCLECI_TOKEN secret and opens or
updates one circleci-nightly-macos issue naming each red job, its job URL and its
artifact names (for example bats-unit-joblog/bats-unit-joblog-durations.tsv).
Without the secret, or if CircleCI refuses it, the run skips with a notice.
The nightly sweeps build unsigned (the CircleCI executor pins signing off); the
GitHub sweeps signed when certificates were configured. Release and signing
workflows (build-ctrl-proxy-ios-ipa, build-network-filter-probe,
build-prototype-agent, build-screen-capture-helper, the desktop installers)
stay on GitHub, the build-* ones on hosted xcode-27 with Xcode 27.2 selected
explicitly.
Self-hosted Mac (#11011)¶
The owner’s Mac (ARM64, 16 cores: 12 performance + 4 efficiency, 128 GiB) runs
two runner pools. A job goes to its pool only when all of these hold;
otherwise, including on every fork PR, it runs on hosted macos-26 /
macos-latest:
- the PR is owner-authored (
github.event.pull_request.user.login == 'kaeawc') from the same repository; - the repository variable
AUTOMOBILE_MAC_POOLS_ENABLEDistrue(the manual override: any other value sends every Mac job to hosted runners); - the
Mac Pool Routingjob did not outputhostedfor that pool.
A job labelled for a pool with no online runner queues forever, holding its required check, so the routing job checks the pools when every run starts.
Mac Pool Routing (#17580). An advisory Linux job (ubuntu-latest, no
needs, no job if) runs scripts/ci/mac-pool-routing.sh. It lists the
repository’s self-hosted runners via the REST API and outputs small
(automobile-mac) and heavy (automobile-mac-heavy), each self-hosted or
hosted. A pool is hosted when no runner carrying exactly that label is
online; a busy online runner still counts, since the job waits for it. It
routes only the current run and never changes the repository variable. Every Mac pool job needs it and leaves its pool only on an
explicit hosted, and runs under always() so a cancelled or failed routing
job does not skip it.
GITHUB_TOKENcannot list runners, so the job reads the repository secretMAC_RUNNER_READ_TOKEN: a fine-grained token for this repository with Administration: read (or a classic token withreposcope). The job’s own permissions are onlycontents: read, so it does not get the workflow’s write scopes alongside that secret.- It fails open to self-hosted. A missing secret, a failed or timed-out (30 s)
API call, or an unreadable response outputs
self-hostedfor both pools and puts a warning and the reason in the job summary. Both steps arecontinue-on-error, and an empty output also means self-hosted, so the routing job cannot skip or misroute a required check. When it fails open, an offline pool queues as before andMac Runner Queuereports it. - Runner status is read once, at run start; a pool that goes offline later still queues its jobs.
- Fallback cost: an iOS run on hosted macOS uses about 18-20 macOS minutes, and hosted jobs count toward the 40-concurrent-job cap. The job summary repeats this on every fallback.
| Pool | Jobs |
|---|---|
automobile-mac |
Swift Code Coverage, Build Desktop App (macos-latest), Installer Minimal (macos-latest), Swift Packages, Swift Prototype Apps, Build Root SPM Package |
automobile-mac-heavy |
Build Xcode Projects (Playground, CtrlProxy), Prototype Simulator iOS 26, iOS Playground Tests (label-forced fallback only) |
- Installer Minimal’s macOS leg uses a Mac only when
scripts/install.sh,package.json,android/gradle/libs.versions.toml, or.github/workflows/pull_request.ymlchanged (#11580). Otherwise its no-op leg reports from Linux with the same required check name. The minimal installer installs the published npm package, and reads the checkout’s compileSdk TOML in the initial SDK advisory, before applying the preset, when adb and an SDK path are detected (also non-interactively). The main-push installer remains unconditional. - The four PR SwiftPM build/coverage jobs cache dependencies in
$RUNNER_TEMP/automobile-swiftpm-cacheon the Mac, symlinked from the isolated home’sLibrary/Caches/org.swift.swiftpm. Home cleanup removes only the link, leaving the directory for the cache post-step to save (#11580). The cache action uses the workspace-relative literal../../_temp/automobile-swiftpm-cache: the same literal on every runner process shares one cache entry and must resolve to$RUNNER_TEMP/automobile-swiftpm-cache. Each isolate step warns, without failing, if the directories differ. Hosted runners keep their two cache paths; DerivedData is unchanged. Mac Runner Queueis an advisory Linux watchdog, with a 15 minute queue-age threshold and 30 second polls. It waits for Fast Validation before observing Mac jobs on GitHub-hostedubuntu-latest, which provides gh, jq, and timeout. It watches the self-hosted jobs present while it runs and exits once none is queued. Later Prototype Simulator (needs: ios-xcode-build) is unwatched so Linux is not held for the whole Mac phase; that job is advisory and starts after a heavy-pool job has just run.- A red
Mac Runner Queuenames queued jobs and their pools: that pool has no online runner or is saturated. A pool that was offline when the run started already went to hosted macOS unless Mac Pool Routing failed open. To force hosted macOS, setAUTOMOBILE_MAC_POOLS_ENABLEDto any value other thantrueand re-run the workflow. The watchdog never cancels jobs and no required gate depends on it. API errors remain warnings and observation is bounded to 15 minutes plus two polls; the job’s 20 minute timeout covers it (#11580). - The heavy pool has exactly one runner process, so xcodebuild and simulator
jobs never overlap, and each job owns the simulators it creates. Heavy jobs
have
timeout-minutesof 25 (30 for the Playground fallback). - Build Xcode Projects is one job that builds Playground and then CtrlProxy in
separate named steps (#11580), since two matrix legs would run serially on the
heavy runner and repeat the setup. Prototype Simulator
needsthat job, so the runner serves the build behind the requirediOS Buildgate first; it still runs whatever the build’s result. - The PR-time XcodeGen drift check (#10939) is the
Check XcodeGen Project Driftstep of Build Xcode Projects, so it runs on the heavy lane. - Every moved job isolates
HOME,TMPDIR, the XDG dirs andCFFIXED_USER_HOMEunder$RUNNER_TEMP, checks out withclean: true, and removes the isolated home. - Xcode: one pin,
AUTOMOBILE_XCODE_VERSIONinscripts/ios/xcode_version.sh(27.2). Hosted runners select it withmaxim-lobanov/setup-xcodeon thexcode-27image. Self-hosted runners usescripts/ci/select-self-hosted-xcode.sh, which exportsDEVELOPER_DIRfor that job only and never changes the Mac’s default Xcode. A missing Xcode fails the job. Jobs that compile.metalsources pass--require-metal, which fails fast when that Xcode lacks its Metal Toolchain. Local scripts callselect_pinned_xcode.test/bats/xcode-version-pin.batsfails when a workflow job selects another version. The CircleCI nightly sweeps and the 26.6 entry of the Build Root SPM Package matrix keep Xcode 26.6, the Swift 6.3 floor, verified; both pinDEVELOPER_DIRto that Xcode soselect_pinned_xcodecannot move them to 27.2. A roll-up job keeps the required check name. - Swift Packages builds unsigned on the Mac: its signing flags are false whenever the job is routed there, so no certificate is imported into the owner’s keychains.
- Cache keys of the moved jobs include
runner.environment, so hosted and self-hosted runs never restore each other’s SwiftPM or DerivedData state. - SwiftLint runs on
ubuntu-24.04, not a Mac (#17581). The 0.57.0 Linux binary linkslibicuuc.so.74andlibxml2.so.2, which Ubuntu 26.04 does not ship, so the job does not followubuntu-latest(#17777). It installs the officialswiftlint_linux.ziprelease asset at the version inscripts/swiftlint/swiftlint_linux_pin.sh, and refuses it unless its SHA-256 matches the digest committed there. The Linux pin must equal the Mac pin (SWIFTLINT_VERSIONinscripts/swiftlint/swiftlint_version.sh, used byprepush-ios.shand Mac installs); theswiftlint-pinsFast Validation check and the installer fail when they differ. SwiftLint loadslibsourcekitdInProc.sofrom the Swift toolchain on the runner’sPATH. The job keeps its name and still fails only on error-severity rules. - Required check names are unchanged (
SwiftLint,Swift Code Coverage,Build Root SPM Package,Installer Minimal (macos-latest),iOS Build), so thegreen-mainruleset needs no edit.
Exception: XCTestRunner Simulator Tests (run-ios-sim label or iOS SDK contract inputs). The
lane still runs on hosted xcode-27 for same-repo PRs. Its daemon steps start
an AutoMobile daemon on the default port and install gems and Homebrew packages,
which would collide with the owner’s daemon and tooling. Moving
it needs a dedicated runner user (see owner steps) and an isolated daemon port.
It runs only on the label or on iOS SDK contract-suite inputs, so it awaits an
owner decision.
N for the small pool: 4. Swift package builds and the desktop Gradle build
are CPU-bound, each able to use most cores. Four small jobs plus the heavy lane
leave about one performance core per process and headroom for the owner’s work;
memory (128 GiB) is not the limit. That is five runner processes: the existing
mac runner, three new small ones and one heavy one.
Namespace macOS (#11012)¶
iOS Device Capture to WHEP (ios-device-webrtc) runs on the Namespace profile
namespace-profile-auto-mobile-macos for owner-authored same-repository PRs.
The Namespace trial allows one macOS job at a time (6 vCPU / 14 GB, billed at
10× the Linux rate), and WHEP fires only on WebRTC changes, so it rarely waits
for that slot. The CircleCI copy is gone.
Routing, in order:
- Same-repo PR,
AUTOMOBILE_NAMESPACE_MACOS_ENABLEDistrue(opt-in; set it after creating the profile),NAMESPACE_RUNNERS_DISABLEDnottrueandIOS_WEBRTC_HEAVY_LANEnottrue: Namespace macOS. - Same-repo PR otherwise, with
AUTOMOBILE_MAC_POOLS_ENABLED=true: the heavy self-hosted lane (the documented fallback, for example if ScreenCaptureKit cannot capture on the Namespace VM). - Everything else, including forks: hosted
xcode-27.
Every placement selects the Xcode 27.2 pin, so the Namespace profile needs Xcode 27.2
installed before AUTOMOBILE_NAMESPACE_MACOS_ENABLED is turned on.
The job keeps one capture retry for the known ScreenCaptureKit flake and a
30-minute timeout. On the heavy lane it isolates HOME like the other
self-hosted jobs, but it still starts an AutoMobile daemon and MediaMTX on fixed
ports. Use that fallback only with the heavy runner under a dedicated user.
Job → runner¶
| Job | Same-repo PR | Fork PR |
|---|---|---|
| SwiftLint | ubuntu-24.04 |
ubuntu-24.04 |
| Swift Code Coverage | self-hosted automobile-mac |
xcode-27 |
| Build Desktop App / Installer Minimal (macOS) | self-hosted automobile-mac |
macos-latest |
| Swift Packages (Xcode 27.2) | self-hosted automobile-mac |
xcode-27 |
| Swift Prototype Apps (one leg per simulator app, Xcode 27.2) | self-hosted automobile-mac |
xcode-27 |
| Build Root SPM Package (Xcode 26.6 and 27.2 entries, one roll-up) | self-hosted automobile-mac |
macos-26, xcode-27 |
| Build Xcode Projects (Playground, CtrlProxy) | self-hosted automobile-mac-heavy |
xcode-27 |
| Prototype Simulator iOS 27.2 | self-hosted automobile-mac-heavy |
xcode-27 |
| iOS Playground Tests | CircleCI (heavy lane if label-forced) | CircleCI |
| iOS Device Capture to WHEP | Namespace auto-mobile-macos |
xcode-27 |
XCTestRunner Simulator Tests (run-ios-sim, contract inputs) |
hosted xcode-27 (exception, see above) |
xcode-27 |
| merge.yml: Build Desktop App (macOS), Generate/Build Xcode Projects | CircleCI post-merge, path-gated, coalesced | n/a |
| nightly.yml: macOS BATS/Node, Xcode sweeps, TSan, XCTestRunner Simulator Tests | CircleCI nightly-macos |
n/a |
| Build tvOS/watchOS/visionOS Runner (Xcode 27.2, Advisory) | hosted xcode-27 |
xcode-27 |
Until the owner steps below are done, the self-hosted and Namespace jobs run on their hosted fallback. A self-hosted placement falls back to hosted for any run where Mac Pool Routing finds that pool offline.
The runner row is ios-xcode-27-2.yml, a separate advisory workflow outside the
ruleset that selects 27.2 explicitly (the image default is 27.1)
(devlog).
Owner steps¶
These need a login or a credential.
CircleCI (web app; do not script them against the API)¶
- Project Settings → Advanced: confirm “Enable dynamic config using setup workflows” is On (already required).
- Project Settings → Triggers → Add scheduled trigger: name
nightly-macos, branchmain, daily at 00:00 UTC (the oldnightly.ymlcron), pipeline parameterrun-nightly-macos=true(boolean). Until this exists the nightly macOS jobs do not run anywhere. - Project Settings → Advanced: leave “Auto-cancel redundant workflows” as it is; post-merge coalescing does not rely on it.
- After the first nightly, check that the
xcode: "26.6"image still exists (the sweep fails at VM start if CircleCI retires it). Also check that Build Desktop App (macos) can install JDK 21 and the Android command-line tools on it. Neither has run on CircleCI yet.
Self-hosted Mac runners¶
Run on the Mac as the user that owns ~/actions-runner:
- Keep Xcode 27.2 at
/Applications/Xcode-27.2.0.appand 26.6 (the root SPM floor entry andxcode-selectdefault) at/Applications/Xcode.app. Install 27.2’s platforms and Metal Toolchain once:
export DEVELOPER_DIR=/Applications/Xcode-27.2.0.app/Contents/Developer
for p in iOS tvOS watchOS visionOS; do xcodebuild -downloadPlatform "$p"; done
xcodebuild -downloadComponent metalToolchain
Without it, jobs that compile .metal sources fail at Xcode selection. The
selector never crosses majors (#11422).
- Get a registration token (valid for one hour):
TOKEN="$(gh api -X POST repos/kaeawc/auto-mobile/actions/runners/registration-token --jq .token)"
- Add three small-pool runners (the existing
macrunner, labelsself-hosted,macOS,ARM64,automobile-mac, is the first). For each of2 3 4: make$HOME/actions-runner-small-$i, extract$HOME/actions-runner/actions-runner-osx-arm64-<ver>.tar.gzinto it, copy.pathand.envfrom$HOME/actions-runner, then./config.sh --unattended --url https://github.com/kaeawc/auto-mobile --token "$TOKEN" --name "mac-small-$i" --labels automobile-mac --work _workand./svc.sh install && ./svc.sh start. - Add exactly one heavy runner the same way (
$HOME/actions-runner-heavy,--name mac-heavy,--labels automobile-mac-heavy). It must NOT carryautomobile-mac, or small jobs could land on it and overlap a simulator job. Preferably install it under a dedicated macOS user so its CoreSimulator device set,~/Libraryand launchd services are separate from the owner’s simulators and AutoMobile daemon. Log in as that user once so its LaunchAgent can start. - Check the pools:
gh api repos/kaeawc/auto-mobile/actions/runners \
--jq '.runners[] | [.name, .status, ([.labels[].name] | join(","))] | @tsv'
Expect mac, mac-small-2..4 (automobile-mac) and mac-heavy (automobile-mac-heavy), all online.
- Turn the routing on:
gh variable set AUTOMOBILE_MAC_POOLS_ENABLED --body true. To force hosted runners, set it to anything else (gh variable set AUTOMOBILE_MAC_POOLS_ENABLED --body false). Runs created after that use hosted runners for every Mac job; jobs already queued for the Mac stay there until their run is cancelled (for example by a new push). - Enable the automatic per-run fallback: create a fine-grained personal access
token scoped to
kaeawc/auto-mobilewith Administration: read and add it as the repository secretMAC_RUNNER_READ_TOKEN(gh secret set MAC_RUNNER_READ_TOKEN). Check it withGH_TOKEN=<token> gh api repos/kaeawc/auto-mobile/actions/runners --jq .total_count. Until it exists,Mac Pool Routingfails open and says so in its summary.
Namespace macOS¶
- In the Namespace dashboard, create a macOS (Apple Silicon) runner profile
named
auto-mobile-macos, so jobs can requestnamespace-profile-auto-mobile-macos. Do this before merging: with the kill switch off, a same-repo WebRTC PR otherwise waits for a missing profile. - Run one WebRTC PR (or add the
webrtclabel) and confirmiOS Device Capture to WHEPcaptures on the Namespace VM (ScreenCaptureKit needs a window server session; the job’s simulator-window probe fails loudly if it is missing). - If it cannot capture, set
gh variable set IOS_WEBRTC_HEAVY_LANE --body true(heavy self-hosted lane) and record why in #11012. - Check the macOS minute cost against the trial/plan on the Namespace usage page.