Est.

Cache Warmth and Test Parallelism as a Reachability-Aware CI Tradeoff

Shrinking the test set first changes how to spend on caches and shards.

Sponsored by Graphide

The Architecture Graph Editors
· 9 min read · Updated
Cover illustration for “Cache Warmth and Test Parallelism as a Reachability-Aware CI Tradeoff”
CI Reachability and Test Selection · November 11, 2026 · 9 min read · 1,954 words

Warm caches and parallel tests make CI faster by reducing different costs. Warm dependency and build caches reduce preparation work, while parallel tests shorten execution after compilation finishes. Reachability-aware test selection changes the tradeoff by shrinking the test set first, so cache costs, shard counts and runner sizes must be judged against the tests a change can reach.

How can you tell whether runner startup, cache restores or test execution is slowing CI?

A slow pipeline costs more than the waiting time because it interrupts the developer’s work on the change. That delay needs a more precise diagnosis. The core workload has three sequential phases, which are dependency installation and build-cache population, compilation of changed code, and test execution. The dominant phase varies by codebase, so a remedy that works for one repository may leave another’s main delay untouched.

Runner startup and cache restoration also take time before the test results arrive. To distinguish those costs, the useful measurement separates startup and restore time from dependency work, changed-code compilation and test execution. A long restore is not a slow test. Nor does adding test shards shorten a serial compile step that must finish before those tests can begin.

Cache warmth reduces dependency installation and rebuilding of prior artifacts, while parallelism distributes test execution. Runner sizing affects both, but it has particular relevance when changed-code compilation dominates. The practical principle is to remove repeated preparation work before distributing tests, then address the remaining compile cost rather than expecting concurrency to solve every phase.

What cache warmth actually means at the runner layer, and why pipeline configuration alone cannot deliver it

A cache action in workflow YAML does not establish that a job starts with its dependencies available. Warmth is a property of the runner’s state. When dependencies and build artifacts are already on the machine, it can skip work that an empty runner must do, even if that runner has a correctly configured restore step. A warm runner with a cold cache still pays installation cost, while a cold runner with a warm cache still pays startup cost.

translationCore4 PR #926, from November 2026, documents the distinction through merge-queue jobs that printed “No cache found”. Pushes to main skipped tests already run by the queue, so main never populated the test cache those jobs needed. The fix gave three Linux test jobs one shared cache entry keyed to the default branch, which queue runs restored but did not save. A nightly job populated that entry independently of test execution.

Cache-key discipline remains a separate requirement. Keys should hash the lockfile or dependency manifest so that invalidation follows dependency changes rather than forcing unnecessary rebuilds or serving stale dependencies. Graphide CI treats warmth as a runner property by running existing GitHub Actions jobs on its runners with dependency and build caches already present. The workflow does not need to recreate that warm state on every run, but precise cache keys are still necessary.

secator PR #1564, from October 2026, shows how broad the cached footprint can be. It includes binaries, source-built tools and vulnerability databases. The first run on a new branch is substantially slower because its cache is cold, while subsequent pushes are warm. A comparison that ignores this distinction can mistake cache preparation for a difference in execution speed.

How parallelism accelerates test execution but cannot fix the phase before it

Matrix builds and pipelines structured around job dependencies allow independent jobs to run concurrently. That helps once independent work is ready to execute. It does not accelerate the serial compilation that precedes the tests, and every additional shard can introduce startup and cache-preparation work. The gain therefore depends on how much executable work each shard receives.

Random or round-robin assignment can leave one shard running substantially longer than the others, making that long tail determine completion time. Aggressive concurrency can also expose shared database records, global singletons and other state that sequential execution leaves undisturbed. Per-shard isolation is needed before teams can count on those gains.

Warm caches can improve the economics of that distribution. For example, Graphide CI’s faster GitHub Actions runners combine warm caches with test splits, made only where a check shows every result stays the same, to target the per-shard preparation cost that can consume concurrency gains. Isolation and balance remain the team’s responsibility, and improvements depend on the workload and its baseline. Parallelism spreads the full suite across runners, but it does not shrink the suite.

What reachability-aware test selection changes about the cache-parallelism tradeoff

Reachability-aware selection changes the amount of work before execution speed enters the calculation. It maps a change to the tests that can reach it, then applies parallelism to that smaller set. The allocation question therefore changes. It is no longer how many shards the repository warrants in the abstract, but how many shards the reachable test set warrants for this change.

For a small reachable set, shard startup and result aggregation may cost more than executing the selected tests. A ten-test run does not justify the same distribution as a thousand-test suite. Cold-cache preparation must also be evaluated against the smaller workload, rather than treated as an acceptable expense because it was economical for the full suite. Selection can cut preparation work when less is needed, but any fixed cold-start cost that remains takes a larger share of the run.

The mapping must be conservative enough not to omit a test that would catch a regression. Uncertainty should mean running the affected tests rather than assuming they are irrelevant, and a failing test must still fail the job. This same-verdict requirement confines selective execution to PR feedback, while the default branch and release validation retain full-suite runs.

How the nightly cache-warm pattern and advisory shadow mode reflect this tradeoff in practice

The nightly cache-warm pattern separates producing a cache from consuming it. In PR #926, the nightly job runs gates.sh --step cache-warm, which compiles without executing the suite. Only the default branch saves the shared entry, while merge-queue jobs restore it. This means the cache is ready even if a main-branch test run never happens, and branch-keyed writes do not use up repository cache quota.

If test jobs begin cold only because earlier workflows skipped cache population, their timing does not cleanly show the cost of the selected workload. With a dedicated warm job, the cache is available even when test policy skips the tests.

Advisory shadow mode addresses the other uncertainty. An advisory file-test-impact job runs on every PR to compare the current module planner’s choices with per-file import reachability, but it does not fail the PR or act as a gate requirement. Teams can compare proposed selections with full-suite results before enforcement, exposing over-selection and under-selection as observable data. Reviewers must also be able to see what ran, what was skipped and why, because narrower execution requires clearer evidence.

Where runner hardware fits into this system, and when it deserves priority

Runner hardware deserves attention after warmth and selection are correctly configured. It can accelerate remaining work, but cannot repair an absent cache entry or an over-scoped test run. Changed-code compilation may still take the most time once those issues are fixed. In that case, runner capacity addresses a real bottleneck.

GitHub’s generally available 96-vCPU larger runner sits at the top of its Ubuntu x64 range, which extends from 2 vCPU/8 GB to 96 vCPU/384 GB. ARM64 variants extend through 64 vCPU. A small reachability-selected workload may use little of a large runner’s capacity, so runner size must fit the selected work even if the repository’s full suite needs more.

CPU count is only part of the comparison. Cache I/O speed also differentiates runner fleets, particularly where restoring or reading cached artifacts dominates preparation. Graphide CI combines runner-resident caches with splitting a job’s tests across runners where a check shows every result stays the same, while every test still runs, making it one option for treating warmth and allocation together. The hardware decision remains distinct from workflow configuration, and larger machines belong after the phase and selection questions have been answered.

Why agent-written code raises the stakes for all three levers simultaneously

Coding agents such as Claude Code, Codex and Cursor increase the pressure on verification by making continuous submissions practical. They are complementary development tools. The constraint moves from writing code toward specifying and verifying it. As more jobs run, repeated cold preparation, full-suite execution that is not needed and too much sharding use up more of the capacity to give feedback.

Test quality adds a separate verification constraint. Agent-generated tests can have reliable structure but weak or absent oracle logic. Test-file presence is not evidence of a verified outcome. A reachability-selected run containing assertionless tests can report success without demonstrating that the change is safe, even if those tests genuinely reach the changed code.

Oracle-aware CI checks can flag newly added test files that lack assertion patterns. This separates structural scaffolding from contributions that verify behavior. Graphide review addresses the related review layer with a pull request check that reports findings and says for each how it is known, including when it could not be verified.

Faster feedback cannot make up for assertions that prove nothing. Strong assertions still need to run in time to give feedback as more submissions arrive.

The limits of static reachability and what they mean for when to fall back to the full suite

Static dependency analysis genuinely misses reflection, dynamic dispatch and runtime-wired dependencies. A small change to a shared-helper can affect behavior outside the traced graph, while a large change to documentation may need no product tests. Change size is therefore a poor substitute for reachability. Nor can a static graph be treated as a complete description of runtime behavior.

Agent-authored code and tests can compound this limitation. A selector may over-select tests that touch no meaningful behavior while missing tests for behavior reached through reflection. Line coverage cannot show whether a test that ran could catch a regression. Mutation testing is the most established mechanism for distinguishing execution from detection, and a reachable test without meaningful assertions is not a safeguard.

Shadow mode provides a way to calibrate policy against this uncertainty before selection becomes binding. Comparing proposed omissions with full-suite results can expose where the graph needs conservative treatment. When reachability cannot be established, affected tests must run, with a full-suite fallback when the change cannot be safely bounded. Release and periodic validations should always run the full suite, keeping selective execution within the PR feedback loop.

How to sequence the three levers when optimizing a real pipeline

For a team optimizing a real pipeline, the sequence begins with auditing actual cache warmth when jobs run. The presence of a cache action is insufficient. The audit should distinguish runner startup, restoration, dependency installation and changed-code compilation from test execution, then establish whether PR jobs consume an entry that another job reliably populates. A nightly cache-warm job can fill the cache those jobs expect when main-branch workflows never do.

The next decision is the safely reachable test set. Selection must precede shard allocation because it changes how much execution remains worth distributing. Parallelism can then split independently executable tests, with isolation for shared state and measured shard balance. A ten-test reachable set should not inherit the runner allocation chosen for a thousand-test full suite.

Larger runners come last, when warmth and selection are correct but remaining compile or test time still exceeds the feedback threshold. Full-suite execution remains required after merge on the default branch, for releases and periodic validation, and whenever reachability cannot safely bound the change. As agent-generated commit volume grows, sequencing becomes more consequential because each new submission repeats either the pipeline’s avoided work or its accumulated waste.

Sources

  1. GitHub Actions hosted runner fleet now includes 96 vCPU larger runner - GitHub Changelog
  2. How Graphide CI works · Graphide docs
  3. Which tests run, and why · Graphide docs
  4. Read a run · Graphide docs

More in CI Reachability and Test Selection