Over the course of this session, the "Pre-publish benchmark gate" and "Perf canary (incremental tiers)" workflows have repeatedly failed the "1-file rebuild" regression check on multiple, mutually-unrelated PRs (#1840, #1843, #1849), each time reporting the "dev" build at roughly 2x the stored baseline:
In every case:
The tight clustering of "dev" measurements (~176-270ms, consistently ~2x the 117ms baseline) across totally unrelated code changes suggests the 117ms baseline was captured under different (faster) runner conditions than what's currently provisioned, rather than random per-run jitter. NOISY_METRICS in tests/benchmarks/regression-guard.test.ts already widens tolerance for this exact metric to 50% (100% in BENCH_CANARY mode) for known runner jitter, but that's proving insufficient against the current baseline.
Suggest: capture a fresh baseline measurement for "1-file rebuild" on current CI hardware and compare against the existing 117ms figure to confirm whether it needs updating, or investigate whether GitHub Actions runner specs/pool changed recently.
Over the course of this session, the "Pre-publish benchmark gate" and "Perf canary (incremental tiers)" workflows have repeatedly failed the "1-file rebuild" regression check on multiple, mutually-unrelated PRs (#1840, #1843, #1849), each time reporting the "dev" build at roughly 2x the stored baseline:
In every case:
src/features/check.ts, a CI-gate predicate unrelated to the build pipeline).RUN_REGRESSION_GUARD=1 npx vitest run tests/benchmarks/regression-guard.test.ts -t "native engine") passes cleanly every time.gh run rerun --failed) eventually clears it (needed 3-4 reruns for fix: recompute directory structure metrics for affected directories on incremental rebuild #1840 and fix: scope reexportedSymbols to actually-named re-export specifiers #1849).The tight clustering of "dev" measurements (~176-270ms, consistently ~2x the 117ms baseline) across totally unrelated code changes suggests the 117ms baseline was captured under different (faster) runner conditions than what's currently provisioned, rather than random per-run jitter.
NOISY_METRICSintests/benchmarks/regression-guard.test.tsalready widens tolerance for this exact metric to 50% (100% in BENCH_CANARY mode) for known runner jitter, but that's proving insufficient against the current baseline.Suggest: capture a fresh baseline measurement for "1-file rebuild" on current CI hardware and compare against the existing 117ms figure to confirm whether it needs updating, or investigate whether GitHub Actions runner specs/pool changed recently.