Each run uses Explorer then Consolidator (optional video review can run between them). The Suite Runner executes that pipeline in parallel across every booted simulator.
The outer loop: xqa run reads a .suite.json, provisions
simulators, and distributes specs through a FIFO worker pool. Each simulator lane runs the full agent pipeline —
shown in miniature here, detailed further down.
A .suite.json file defines what to run — spec files and/or freestyle
explorers.
{
"specs": [
"specs/payments/*.test.md",
"specs/auth/*.test.md"
],
"freestyle": 2
}
Glob patterns resolving to .test.md spec files. Each matched spec runs the full pipeline
(Explorer → Consolidator) on its assigned simulator. Duplicates across patterns are removed.
Number of freestyle explorer agents to run. Each gets its own simulator and performs a breadth-first UI sweep without spec files. Useful for catching regressions outside of scripted flows.
Both fields are optional but at least one must be present. Specs and freestyle agents compete for simulators through the same FIFO worker pool.
Zooming in on one simulator lane: Explorer runs first, then Consolidator — each stage strictly sequential.
Drives the iOS simulator, captures screenshots & accessibility trees, detects navigation and visual regressions, records video.
The Explorer is the first agent in the pipeline. It autonomously navigates the app in the iOS simulator,
interacting with UI elements while capturing screenshots and accessibility tree snapshots at each step. It
operates in two modes: freestyle (breadth-first sweep) or spec-guided (following markdown test specs). It
records the entire session as video for later analysis. Visual quality review is delegated to a separate
visual-pass hook — an isolated single-turn Anthropic call that fires synchronously after every
view_ui capture and emits visual findings (design-system-violation,
visual-regression, interaction-regression) via a forced visual_scan
tool. The hook has no spec / navigation / app context, which empirically removes the tunnel-vision failure
mode where the agent in spec mode skips visual review while focused on assertions. The hook is gated by the
visualMode config: disabled skips the hook entirely,
enabled-no-designs runs the visual reviewer with no artboard reference, and
enabled-with-designs additionally surfaces candidate design artboards alongside each
view_ui result. Both paths write findings into the same store.
ExplorerConfig — mode, app context, device UDID, timeout, visualModeagents.explorer.visual.designsDir when visualMode is
enabled-with-designsExplorerArtifacts — findings, video paths (1x/2x/4x), snapshotsDeduplicates findings from all agents, applies dismissal filters, outputs final report.
The Consolidator is the terminal agent. It receives all findings from the Explorer. It sends them to Claude
for intelligent deduplication (same bug reported by multiple agents gets merged), applies persisted
dismissals (previously marked false positives), and writes the final findings.json.
findings.json — deduplicated, filtered final findingsfindings.json to diskFive components collaborate to orchestrate parallel spec execution.
CLI entry point. Reads .suite.json, resolves spec globs and freestyle agent
count, validates config, discovers simulators, orchestrates provisioning, creates the worker pool, and handles
SIGINT gracefully.
The Run Command is the top-level orchestrator. It reads a .suite.json config file that defines
which specs to run (via glob patterns) and how many freestyle explorers to dispatch. It resolves globs into
spec file paths, checks that the app path is configured (IOS_APP_PATH or
ANDROID_APP_PATH), and discovers booted simulators. Spec runs and freestyle runs
are both enqueued into the same FIFO worker pool. Progress is shown in the terminal; on completion it writes
suite findings. On SIGINT it captures partial results and exits with code 130.
.suite.json — specs (globs) + freestyle countIOS_APP_PATH / ANDROID_APP_PATH, QA_EXPLORE_TIMEOUT_SECONDSSIGINT handler for graceful abort (exit 130)
Distributes spec execution across simulators. Per-spec failures are captured — not thrown. Handles timeouts with simulator re-provisioning.
The Worker Pool maintains a FIFO queue of pending specs. Each simulator worker claims the next spec when it becomes free. It runs the full pipeline (Explorer → optional video review → Consolidator) for each spec. Failed specs are isolated so one failure never crashes another simulator's work. On spec timeout, the simulator is re-provisioned before accepting the next spec.
SuiteObserver progress callbackcompleted / failed / timeoutSuiteEvent streamRenders suite progress. TTY mode: interactive dashboard with zoom (1-9 per spec, ESC to return). CI mode: tagged greppable log lines.
SuiteDisplay adapts to interactive terminals vs CI. In TTY mode it renders an interactive dashboard: spinners for running specs, checkmarks for completed ones, and a live queue section. Press 1-9 to zoom into a spec's verbose output; press ESC to return to the overview. In CI mode it emits single-line tagged log entries that are easy to grep. Terminal state is always restored on teardown.
SuiteObserver interfacefinally and
process.on('exit')Aggregates per-spec findings into a single output file. Written atomically after all specs
complete. On abort, partial results are written with "aborted": true.
The Suite Findings Writer collects per-spec pipeline results and writes suite-findings.json to
a deterministic path. The suiteId is derived from sorted, slugified glob patterns. The
runId is a zero-padded counter incremented per suite-plus-date combination. Findings are
attributed per-spec with no cross-spec deduplication — each spec's findings remain independent.
suite-findings.json at deterministic pathsuiteId derived from sorted + slugified glob patterns
runId incremented per suite + date combination"aborted": trueManages simulator lifecycle. Discovers booted simulators, then provisions each: terminate → install → launch. Idempotent. Re-provisions after spec timeout.
Simulator provisioning installs and launches your app on each booted simulator before work begins. Discovery finds simulators that are already running. Provisioning terminates the app before installing so a stale binary is never left behind. After a timeout the pool provisions again before the next spec.
IOS_APP_PATH / ANDROID_APP_PATH)What you see while a suite runs — each item moves through these stages.
Capabilities you rely on during a run — not internal package names.
Shared shapes for findings, run artifacts, and reports used across Explorer and Consolidator.
Live suite progress in an interactive TTY dashboard, or greppable tagged lines in CI.
Drive the simulator — view UI, tap, screenshot, and related actions during exploration.
Orchestrates Explorer, optional video review, and Consolidator into one run with retries and recovery.