9. API-driven self-play test harness

The automated self-play harness is a test-only layer in rts_ai::selfplay. It is intentionally separate from the simulation core: gameplay AI is a player feature, while self-play is a regression harness for exercising the public simulation API.

Contract. Self-play scripts may only drive the game through the Game seam in §3.1: start_payload(), snapshot_for(player), enqueue(player, SimCommand), tick(), alive_players(), and tick_count(). Scripts observe the same fog-filtered snapshots a client would receive and issue ordinary domain commands. They must not mutate entities, players, map state, or private system internals. This keeps the simulation architected for future API clients, replay tools, and external test drivers without adding a second privileged control path.

Command log replay. Game records every command at the authoritative apply tick, after callers have enqueued human, scripted, or AI commands and before systems apply the pending queue. game/replay.rs translates that wire-compatible log into SimCommands, restores a fresh Game from the replay artifact’s checkpoint-backed start composition with AI thinking disabled, and compares the resulting event stream and final per-player snapshots. Replay and live play use the same typed command application path, so a replay proves both the recorded command artifact and the deterministic simulation ordering. Preserved schema 2 incident replay files are historical evidence, not supported replay fixtures; current replay workloads use schema 3 captures. Entity iteration and A* tie-breaking must remain stable; avoid hash-order-dependent simulation behavior.

Derived-state rebuild coverage. The test-only Game rebuild seam clears the persistent PathingService cache and rebuilds the final spatial index from authoritative entities at a tick boundary. The paired-game derived-state test warms pathing, rebuilds derived state in one copy, and then compares semantic authoritative state plus player, full, and spectator snapshots while both games continue ticking and repathing.

Fast/full AI split. Plain cargo nextest run --config-file .config/nextest.toml --manifest-path server/Cargo.toml --profile default keeps the self-play harness in the default gate, but only runs the fast scripted coverage. Long profile-backed and real-AI self-play tests return early unless RTS_FULL_AI_TESTS=1 is set; tests/run-all.sh --full-ai enables that mode for the full orchestrator. RTS_SELFPLAY_FULL=1 remains accepted as an alias for manual self-play runs. Use full AI coverage when touching AI strategy, profile-backed self-play, replay determinism, or balance behavior that depends on long matches.

AI arena runs that end with no winner because of elimination remain unresolved elimination draws; they are not scored using the tick-cap army-value tiebreak. An arena run is rejected when distinct candidate and baseline requests resolve to the same concrete profile for a seed.

Profile-backed coverage. The long profile-backed tests spawn AI-profile players through the self-play adapter and run matches headlessly under RTS_FULL_AI_TESTS=1 cargo nextest run --config-file .config/nextest.toml --manifest-path server/Cargo.toml --profile default. The profiles gather steel and oil, construct supply and tech structures, train Riflemen, Scout Cars, and Tanks, and launch attack-move waves at public enemy start tiles. The self-play adapter owns harness-only state such as pending build intents, failed build spots, and staging/attack guards needed to interpret fog-filtered snapshots without duplicating profile strategy logic. The harness checks per-tick invariants for invalid resources, supply overflow, malformed entity snapshots, out-of-bounds positions, and non-finite progress values. It also enforces progress deadlines so a stuck economy/tech/combat loop fails as a deadlock instead of timing out silently.

Special harness scripts remain where they cover behavior that is not a normal AI strategy profile: WorkerRushScript is an all-in worker-pull scenario, and MineOnlyScript is passive mining/fairness coverage. These scripts are kept isolated from the canonical profile list.

Artifacts. On failure, the test writes target/selfplay-failures/<test>-<pid>-<time>/ with:

The replay artifact is meant to be enough to reproduce or inspect a failing run without manually playtesting first. Load an artifact with /?replayArtifact=<artifact_name> on a local server using the same Cargo target directory. The older /dev/replay-artifact?replay=<artifact_name> route redirects to that canonical launch URL. For DB-free group replay lobby coverage, POST /dev/replay-lobby?replay=<artifact_name> loads the same safe artifact directories and returns a __match_replay__:* staging room without exposing the artifact JSON; the normal saved-artifact URL still uses immediate replay confirmation/playback. By default successful runs do not write artifacts. For manual inspection, setting RTS_SELFPLAY_SAVE_REPLAY=1 writes a successful run to target/selfplay-artifacts/<test>-<pid>-<time>/; setting RTS_SELFPLAY_SAVE_REPLAY=<name> uses that explicit safe artifact name instead.

Profile matchup CLI. The ai-matchup binary is the manual fixed-horizon matchup facility for profile-vs-profile runs. It composes the same self-play adapter and Game seam as the tests, runs one directed match until a starting City Centre objective win or the tick cap, optionally verifies deterministic replay, and can write a replay artifact. No winner by the default 25,000-tick horizon is a draw; army value and other material metrics are diagnostics, not tiebreakers:

cd server
cargo run --bin ai-matchup -- ai ai
cargo run --bin ai-matchup -- ai_2_1 ai_turtle --seed 7 --ticks 3000 --json
cargo run --bin ai-matchup -- default ai_turtle --seed 7 --ticks 25000 --json
cargo run --bin ai-matchup -- --list-profiles

The ai-arena binary is the agent-facing profile comparison layer. It runs side-swapped seed pairs around the same profile matchup result, defaults to AI 2.1 against AI Turtle, and writes a top-level arena-summary.json plus per-run sidecars:

cd server
cargo run --bin ai-arena -- --candidate ai_2_1 --baseline ai_turtle --seeds 3 --ticks 9000
cargo run --bin ai-arena -- --candidate ai_turtle --baseline ai_2_1 --out-dir target/ai-arena

Each run directory contains a deterministic replay.json plus manifest.json, summary.json, decision-trace.jsonl, and brief.md. The manifest records canonical profile identities and fingerprints; the brief is the first artifact agents should read before opening the replay or searching trace labels.

Keep fast invariant-style milestone coverage in cargo nextest run; use RTS_FULL_AI_TESTS=1 cargo nextest run --config-file .config/nextest.toml --manifest-path server/Cargo.toml --profile default for the long regression gate and the CLI for balance exploration, seed sweeps, and strategy result sampling.

10. Dev scenario inspection

Game-backed dev scenarios are live, no-fog watcher rooms for inspecting authored simulation situations through the normal Pixi client. Start a local server, then open the index:

open "http://localhost:<port>/dev/scenarios"

The index lists every supported launch and links to the current URL shape:

/dev/scenarios?id=<scenario_id>&unit=<unit>&count=<count>[&blocker=<unit|none>][&case=<case>]

The handler redirects into the normal client with watchScenario=1; the client auto-joins a reserved spectator room named:

__dev_scenario__:<scenario_id>:unit=<unit>:count=<count>[:blocker=<unit|none>][:case=<case>]

Current scenario ids:

The watcher shows movement debug path overlays by default. Replay speed controls are reused for dev scenarios: Pause sets the simulation speed to zero, and Step advances exactly one authoritative tick while paused. Normal seek/reset controls are replay-only.

Scenario setup is server-side only under server/crates/sim/src/game/setup/dev_scenarios.rs; do not expose arbitrary spawning or map editing through client commands. The Interact dev-scenario namespace may observe, frame, screenshot, record, and time-lapse these watcher rooms; its artifacts remain confined under target/interact/scenario/<session-id>/.

The Tank Trap pathing matrix scenarios are harnesses for owner-independent vehicle pathing, infantry pass-through, explicit infantry attacks, and attack-move acquisition filtering. Vehicle path planning, physical movement, and standability treat every live Tank Trap footprint and closed one-tile gap as a vehicle-body blocker regardless of ownership or visibility. Combat acquisition should prioritize a Tank Trap only when that trap lies on the vehicle’s bounded route window or closes a gap across it; irrelevant nearby traps should lose to combat targets.

11. Package-aware test selection policy

The authoritative full gate is the PR ./tests/run-all.sh check from the Main test gate workflow. Local runs should usually be narrower and selected by the changed files or contracts. Use node tests/select-suites.mjs --from=<base-ref> or pass changed paths directly to see the expected suites.

Lab coverage derives the expected lategame research set from the Kriegsia catalog and requires every bundled preview scenario to grant that full set. Client fixtures treat completed research arrays as unordered state. The lab client contract suite requires pasted JSON with the retired labScenario envelope to fail locally with an explicit lab result instead of falling through to generic server message parsing. The agent lab driver exposes its page bridge only on Lab routes, rebuilds the server for the selected worktree, cancels interrupted startup deterministically, and transfers daemon startup to the child through a random nonce lease before socket bind. Detached daemon startup errors surface immediately, partial startup ownership and runtime resources are cleaned up, and idle expiry uses monotonic time. Dead-parent locks are preserved for a bounded grace period, only verified stale records are reclaimed, and lock release requires nonce ownership. Malformed bridge inspection filters fail closed so invalid bounds cannot broaden a query. The driver evaluates capture fallback and render errors against the current frame so transient startup work does not block a later clean capture, and clamps seeks to the supported range.

Lab preview contracts require a 24-hour-or-longer artifact TTL and prove that a published URL remains fetchable after the Lab publisher closes and its originating worktree is removed. CLI coverage also proves daemon shutdown leaves issued screenshot links valid, while concurrent recording wait/stop callers receive the same deduplicated durable publication.

Interact source, page-bridge, Rust artifact-bridge, focused-test, CLI-documentation, and local skill changes select interact-contracts plus the browser client-smoke shard. The fast Node/static gate installs the root lock and runs the strict no-emit Lab TypeScript check before the contracts. Node-side Lab implementation sources execute directly as .ts on Node 22.18+, while the browser bridge and tests remain buildless JavaScript. The fast contracts use the fake driver and isolate UUID-named session and portable artifacts, while the one live canary exercises open, spawn, update, order/step, screenshot/PNG preview, setup round trip, short H.264 recording, reset, close, stale-session rejection, and daemon shutdown. Standalone live canary runs own a private server; the browser shard passes its existing loopback server through RTS_INTERACT_LAB_BASE_URL so CI does not build or start a second one.

scripts/check-source-file-sizes.mjs runs as a cheap policy gate and enforces a 1500-line cap for Rust, JS, and MJS source/test files under server/, client/src/, tests/, and scripts/, plus the checked-in production stylesheet at client/styles.css. Files that were already above the cap are frozen in scripts/source-file-size-baseline.json; new over-cap files fail, and frozen exceptions fail on growth. Shrinkage prints a ratchet note so the baseline can be lowered or removed.

scripts/check-crate-boundaries.mjs is part of the gate and fails on forbidden Cargo package edges or server-only imports in lower crates. The sim architecture ratchet is also part of the gate: cargo run --manifest-path server/Cargo.toml -p rts-archcheck -- check-sim-architecture fails when rts-sim::game grows new service edges, broad mutable APIs, direct state writes/usages, public API surface, or file-size budget over the committed baseline. Prefer reducing coupling first. If the growth is intentional, update server/crates/archcheck/baselines/sim-architecture.json with:

cargo run --manifest-path server/Cargo.toml -p rts-archcheck -- check-sim-architecture --bless --reason "short reason"

Avoid broad allowlist additions unless the same change or a tracked follow-up explains the cleanup path. tests/select-suites.mjs --verify keeps the changed file mapping itself covered by small examples. CI comments document any intentionally skipped suite; that skip becomes invalid when the changed-file mapping selects the skipped behavior.

12. PR CI contract

The canonical required PR check context is ./tests/run-all.sh in the Main test gate workflow. It is an aggregate check over split coverage jobs for server binary build, Rust policy/lint, two complementary Rust nextest partitions, live Node, and two complementary browser/tri-state shards on pull requests targeting main and on pushes to main. The two nextest jobs use slice:1/2 and slice:2/2, so their union runs the same Rust test set as the local gate. The browser shards divide the current PR coverage into client smoke plus phase 0.5, and phases 2.5 plus 5; each shard gets an isolated prebuilt server. The split jobs run tests/run-all.sh sub-modes under CI so the required aggregate gate preserves client smoke plus tri-state browser coverage without serializing every suite in one runner. The server-build job uploads generated sim-WASM browser assets, and both browser shards download them into client/vendor/sim-wasm before client smoke runs from its clean checkout. Local tests/run-all.sh runs keep client smoke in the default browser gate but skip the latency-sensitive tri-state browser scenarios unless --with-tri-state-browser or RTS_RUN_TRI_STATE_BROWSER=1 is set. WASM-backed tri-state groups also stay opt-in unless RTS_RUN_WASM_TRI_STATE=1 is set. When the generated prediction WASM glue is absent, the server serves a JavaScript fallback module that disables prediction without a 404; generated WASM files take precedence, and other missing assets still return 404. Client smoke reports failing response URLs with browser console resource errors. Changed-file detection classifies PRs and main pushes as docs_only, client_only, or full from the PR base/head range or the push before/after range. docs_only keeps the same check contexts green but exits before expensive suites. client_only is limited to conservative client/ paths and skips Rust nextest, lint, and Rust architecture work while still building the server and running live Node plus browser coverage. Contract-adjacent client paths such as the client/src/config.js facade and the explicitly classified rules, faction, timing, and player-palette mirror modules, along with client/src/protocol.js, client/src/net.js, client/src/lobby_view.js, and generated sim-WASM assets, fall back to full. Client-owned client/src/config/presentation.js remains client-only, and selector verification requires each production config module to declare one classification. Branch protection should require this single aggregate full-gate check unless a plan phase explicitly changes the contract.

node scripts/check-docs-health.mjs runs in the early changed-files CI lane before expensive split jobs. It validates docs/doc-map.json, enforces the 5 KiB docs/context/*.md capsule cap, and checks local Markdown links in docs/ and plans/.

The PR ownership workflow validates owned agent PR metadata for zvorygin/* branches with scripts/check-pr-ownership.sh.

scripts/agent-pr.sh reuses the changed-file policy before opening or updating an owned PR. A supplied --head value must match the current branch before the docs-only skip can push or post status. Before that final classification, it runs the ordered entries in scripts/agent-pr-passes.json; mutating passes must commit their work and leave the same branch clean so the adversarial review covers their final output. Pass reports are preserved in the PR body. When the resulting branch diff against origin/main contains only .md files, including Markdown files outside docs/, it skips the Codex adversarial quality pass but still pushes the branch, posts a successful adversarial-quality-pass status, and writes a docs-only skip report into the PR body. Any non-Markdown changed file keeps the normal adversarial quality pass requirement.

Rust formatting is intentionally not a CI test gate. The repository pins Rust and rustfmt in rust-toolchain.toml; the final quality pass invoked by scripts/agent-pr.sh runs scripts/format-touched-rust.sh after review and before committing/pushing. It formats only Rust files changed by the branch or quality pass, so an owned PR carries its own formatting without a workspace-wide formatter pass or an unrelated formatter failure.

The old standalone Rust and Integration workflows are retired. Their package, architecture, live Node, and browser coverage is owned by the split Main test gate jobs under the required aggregate ./tests/run-all.sh check, so separate auxiliary workflows would duplicate coverage and consume extra runner capacity without increasing merge safety.

GitHub Actions uses standard ubuntu-latest runners for this contract. Public-repository standard runners are acceptable for the current cost posture, while larger paid runner classes are out of scope. The gate remains portable through tests/run-all.sh so it can run locally or on another runner if the hosting or billing posture changes.

PR workflows use concurrency groups scoped by workflow plus PR number, with cancellation enabled only for pull request events. A newer push to the same PR branch may cancel superseded runs, while pushes to main and unrelated branches keep independent results.

Beta deployment is downstream of the full gate but must only deploy tested main push commits. The deploy workflow checks that the completed Main test gate run came from a push event on main before checking out and deploying the tested head SHA.

13. Documentation drift sweeper

scripts/docdrift-sweep.mjs --dry-run is the deterministic operator surface for reviewing commits between docs/docdrift-checkpoint.txt or --base and --head. It reads commit metadata, changed paths, compact diff stats, docs touched, and docs/doc-map.json trace-map candidates, but does not edit docs, create PRs, or advance the checkpoint. Merge commits, empty commits, and docs-only churn are skipped before classifier prompts are built.

scripts/docdrift-sweep.mjs --classify adds the cheap Codex CLI classifier. Live classifier runs must use Codex CLI authentication through the local codex exec path; they must not use the OpenAI Agents SDK, direct API clients, API-key environment variables, or API-billed fallback routes. Fixture runs use --no-codex --fixture <name> and are the required focused verification path before any live Codex smoke. Classifier decisions are cached under the ignored .docdrift/classifier-cache/ runtime directory by prompt version and commit SHA, and reports can be written with --out-dir. Live Codex calls run read-only with approval policy forced to never via Codex config override, emit per-commit progress on stderr, and record token usage when the Codex JSON event stream includes it. Each Codex CLI invocation is wall-time bounded by --codex-timeout-seconds so a wedged classifier or doc-patch generation call fails the run and leaves the daily failure marker instead of blocking launchd indefinitely.

scripts/docdrift-sweep.mjs --generate-docs reruns or reuses the classifier records, selects only update_docs decisions, loads targeted authoritative design-doc sections, and asks Codex CLI for exact minimal find/replace doc patches. The generator prefers classifier-selected design docs; docs touched in the commit and broad trace-map design docs are fallbacks, not an automatic union. It builds and applies doc-patch prompts sequentially so later update_docs decisions see docs already changed by earlier decisions in the same sweep; if the supplied sections already cover the behavior, the generator should return an empty patch set instead of restating it. The script applies generated patches to the working tree and writes docdrift-generate.{md,json} with --out-dir; operators inspect the resulting docs diff before any PR lifecycle step. Fixture runs use the same --no-codex --fixture <name> path and must remain idempotent. If a retry sees that a cached patch’s replacement text is already present, it reports the patch as already applied without spending another Codex generation call. Patch application is atomic per commit: all patches for one update_docs decision are validated against the current working tree before any file is written. If one update_docs decision cannot generate or apply a safe patch, the report records that decision in docPatch.skipped[] with the commit, error kind, and message, then continues with later decisions. Skipped doc-patch decisions do not make the command exit non-zero; they are visible in docdrift-generate.{md,json}, the full-sweep report, and any sweep PR body so humans can decide whether to follow up manually.

scripts/docdrift-sweep.mjs --full is the PR-first operator lifecycle. It fetches origin/main, uses the local checkpoint from .docdrift/checkpoint.txt when present, falls back to the committed seed in docs/docdrift-checkpoint.txt, and gives each new run a unique zvorygin/docdrift-sweep-<run-id> branch plus matching isolated worktree. Before creating that branch it atomically writes .docdrift/runs/<run-id>/run-state.json; the schema-versioned record is updated after every lifecycle step and is the authority for the run’s base/head, generated head, branch/worktree, PR identity and state, checkpoint target, and recovery action. With no explicit --run-id, recovery resumes only the single recorded nonterminal run; multiple candidates fail closed.

An open PR resumes its exact recorded branch, head, PR, and first incomplete step. A missing worktree can be recreated from an exact local or remote recorded head, and the only automatic ref reconciliation is a clean local fast-forward to that exact remote head. Merged runs may finish their recorded idempotent checkpoint step; merged or closed-unmerged terminal runs otherwise keep their old refs and reports while a fresh unique run starts at fetched origin/main. The one legacy fixed branch, zvorygin/docdrift-sweep, can be adopted only when its clean local, remote, and worktree heads agree with exactly one terminal owned PR head. Automatic adoption is restricted to the known 68f6e958... incident; another verified terminal head requires explicit --adopt-legacy, and the legacy ref is never resumed or rewritten. A closed PR’s stale mergeability metadata is terminal evidence, not a conflict decision.

Dirty or conflicted worktrees, in-progress Git operations, non-fast-forward or mismatched heads, open conflicted PRs, ambiguous PR/run matches, and ref or worktree collisions stop before refs, checkpoints, or GitHub state are changed. Recovery never stashes, resets, rebases, force-pushes, deletes, or resolves conflicts. Classification and doc generation then run on the selected safe worktree, scripts/agent-pr.sh owns PR creation, and scripts/wait-pr.sh proves merge reachability. The checkpoint advances atomically only after a no-PR range is fully processed or a recorded merged run is verified. Generated sweep PRs carry the docdrift-sweep label so effectiveness audits can list sweep output separately from PRs that merely change the sweeper tooling. Per-decision doc-patch skips do not block checkpoint advancement: once any generated docs PR merges, or once a run has no docs changes to PR, the checkpoint advances to the processed head. This keeps the nightly gardener from retrying the same stale patch indefinitely. Lifecycle failures such as failed checks, closed PRs, stale branches, dirty sweep worktrees, or unrecoverable git/GitHub errors still exit non-zero and leave the checkpoint unchanged.

Full sweeps write ignored local reports under .docdrift/runs/<run-id>/, including the recovery authority run-state.json, docdrift-full.{md,json}, and any classify/generate reports. The full report’s sweep.recoveryAction states whether the run was created, resumed, completed after merge, started after a terminal PR, adopted from the legacy branch, or stopped for operator review. Use scripts/docdrift-daily.sh as the launchd-friendly daily command; pass normal docdrift-sweep.mjs options after it, for example --dry-run for a lifecycle preview or --run-id <id> for predictable report paths. The wrapper first fetches origin/main, creates or refreshes a clean detached .docdrift/worktrees/docdrift-runner checkout at that ref, then runs that latest scripts/docdrift-sweep.mjs with --repo pointed back at the primary checkout. The primary checkout still owns .docdrift checkpoints, caches, reports, and sweep worktrees, while the runner worktree prevents stale local main from running obsolete sweeper code. Set DOC_DRIFT_RUNNER_WORKTREE to override the runner location. The wrapper defaults scheduled runs to --max-commits 300 so ordinary daily backlog does not trip the interactive classifier guard; set DOC_DRIFT_MAX_COMMITS to override that limit. It also passes --codex-timeout-seconds from DOC_DRIFT_CODEX_TIMEOUT_SECONDS, defaulting to 300 seconds per Codex call. When the daily command exits non-zero, it writes an ignored .docdrift/last-failure.md marker with the command, UTC timestamps, exit code, and stdout and stderr tails, and it clears that marker after the next successful run. The wrapper only runs the command; it does not install or require a launchd job for other developers.

The recurring daily schedule is local macOS launchd, not a GitHub Actions scheduled workflow. Do not start a “did it run?” investigation in .github/workflows; start with the loaded user LaunchAgent. On the primary workstation the job is installed as $HOME/Library/LaunchAgents/com.zvory.rts-docdrift.plist with label com.zvory.rts-docdrift, WorkingDirectory set to /Users/az/Code/rts-0, and ProgramArguments beginning with /Users/az/Code/rts-0/scripts/docdrift-daily.sh. Its StartCalendarInterval is local wall-clock time, currently Hour = 4, Minute = 0. Run artifacts use UTC ids, so a 4 a.m. run may appear as .docdrift/runs/YYYY-MM-DDT08-00-...Z during daylight-saving time and .docdrift/runs/YYYY-MM-DDT09-00-...Z during standard time.

For future operational checks, use this order:

launchctl list | rg -i "docdrift|rts-docdrift"
plutil -p "$HOME/Library/LaunchAgents/com.zvory.rts-docdrift.plist"
ls -lt .docdrift/runs
sed -n '1,180p' .docdrift/last-failure.md
tail -n 120 "$HOME/Library/Logs/rts-docdrift.out.log"
tail -n 120 "$HOME/Library/Logs/rts-docdrift.err.log"
git -C .docdrift/worktrees/docdrift-runner rev-parse --short HEAD
git -C .docdrift/worktrees/docdrift-sweep status --short

Treat .docdrift/last-failure.md as the fastest answer for “did the daily fail?”. A partial full sweep can still create, merge, and checkpoint a successful prefix PR before exiting non-zero; in that case the failure marker is expected and should name the later commit that stopped doc generation. Use the linked PR or the prUrl/sweep.action fields in .docdrift/runs/<run-id>/docdrift-full.json to distinguish a clean no-op, a merged full sweep, a merged partial prefix, and a hard failure that produced no PR.