[Visual Test] Label preflight regressed: gh label list pagination skips run-visual-test on run #19

[MichaelFisher1997]

Workflow run

https://github.com/OpenStaticFish/ZigCraft/actions/runs/29233512374

Trigger: scheduled nightly, commit 16527e3 (feat: add generated preloaded menu preview #915), dev branch.

Symptom

The visual-test workflow fails at step 8 ("Ensure visual-test label exists") and never reaches step 10 ("Run menu screenshot capture"). No screenshot.png is produced and no build-output.log is written. Every downstream step (Setup Lavapipe Vulkan, Run menu screenshot capture, Compare against golden image, Upload screenshot artifact, Upload build log artifact) is skipped. The visual-regression gate is effectively dead until this is fixed.

This is the same failure mode as #916 (run #17 / run #18). The fix proposed there has not landed, so the workflow regressed on the next scheduled run (#19, this issue).

Why build-output.log is missing

The diagnosis environment did not contain build-output.log. That is consistent with the failure: build-output.log is only created by the run-with-log composite action used in the "Run menu screenshot capture" step (visual-test.yml:76-81). Because step 10 was skipped, the tee never ran and no log file was written.

The only artifact from the runner is weston.log, which shows the compositor started normally and was killed 4 seconds later by the post-failure cleanup step:

[07:54:50.732] weston 15.0.1
[07:54:50.737] Command line: weston --socket=headless --backend=headless-backend.so --width=1280 --height=720
[07:54:50.737] Output 'headless' enabled with head(s) headless
[07:54:54.172] caught signal 15

Signal 15 (SIGTERM) at 07:54:54 lines up with step 8 completing in a failed state at 07:54:54 (per the job's steps array), and the always-run "Stop headless Wayland compositor" cleanup step firing immediately afterward. The game binary was never executed, so there is nothing to diagnose in the engine, screenshot path, Lavapipe, or shader pipelines.

Job step status (run #19, attempt 1)

#	Step	Status
1	Set up job	success
2	Run actions/checkout@v4	success
3	Setup Nix	success
4	Start headless Wayland compositor	success
5	Prepare opencode cache	success
6	Load visual verification prompt	success
7	Load visual diagnosis prompt	success
8	Ensure visual-test label exists	failure
9	Setup Lavapipe Vulkan	skipped
10	Run menu screenshot capture	skipped
11	Check screenshot exists	success (exists=false)
12	Compare against golden image	skipped
13	Upload screenshot artifact	skipped
14	Check build log exists	success (exists=false)
15	Upload build log artifact	skipped
16	Stop headless Wayland compositor	success
17	Run opencode visual verification	skipped
18	Run opencode failure diagnosis	in_progress (running opencode here)

Root cause

The pre-flight existence check in .github/workflows/visual-test.yml:55-68 is buggy. gh label list (without --paginate or a higher --limit) returns only the first 30 labels, sorted alphabetically. The ZigCraft repo currently has 41 labels, so run-visual-test is never seen by the check:

$ gh label list --repo OpenStaticFish/ZigCraft --json name --jq '.[].name' | wc -l
30
$ gh label list --repo OpenStaticFish/ZigCraft --limit 100 --json name --jq '.[].name' | wc -l
41
$ gh label list --repo OpenStaticFish/ZigCraft --limit 100 --json name --jq '.[].name' | grep -E '^(visual-test|run-visual-test)$'
visual-test
run-visual-test

visual-test happens to land inside the first 30 alphabetically (sort rank ~21), but run-visual-test (which sorts after the 30-label page cutoff) is hidden. The check therefore believes run-visual-test is missing and runs gh label create "run-visual-test" ..., which fails because the label already exists. gh label create exits non-zero and aborts the step:

2026-07-11T06:59:02.2188083Z if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
2026-07-11T06:59:02.2188445Z   gh label create "run-visual-test" \
2026-07-11T06:59:02.2188798Z     --description "Run deterministic visual regression workflow on a PR" \
2026-07-11T06:59:02.2189127Z     --color "E06C75"
2026-07-11T06:59:02.2189340Z fi
...
2026-07-11T06:59:03.0186758Z label with name "run-visual-test" already exists; use `--force` to update its color and description
2026-07-11T06:59:03.0197803Z ##[error]Process completed with exit code 1.

(That snippet is from #916's run #17; run #19 emits the same gh label create failure.)

Origin

• .github/workflows/visual-test.yml:55-68 — Ensure visual-test label exists step.
• Specifically .github/workflows/visual-test.yml:57 and .github/workflows/visual-test.yml:62 — the unpaginated gh label list --json name --jq '.[].name' calls.

The downstream game code paths were not exercised at all on this run, so the screenshot pipeline (screenshot.zig), the headless swapchain (vulkan_swapchain.zig:119-166), and HomeScreen rendering (modules/game-ui/src/screens/home.zig) cannot be blamed for this failure. If the preflight is fixed the next run should either succeed or surface a different engine-level failure worth investigating.

Suggested fix

Pick one of the following (in increasing order of robustness):

1. Page the label list (minimal change, matches what #916 already recommends)

Add --paginate (or --limit 100) to both gh label list invocations:

- name: Ensure visual-test label exists
  run: |
    if ! gh label list --paginate --json name --jq '.[].name' | grep -q '^visual-test$'; then
      gh label create "visual-test" \
        --description "Issues from automated visual regression tests" \
        --color "E06C75"
    fi
    if ! gh label list --paginate --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
      gh label create "run-visual-test" \
        --description "Run deterministic visual regression workflow on a PR" \
        --color "E06C75"
    fi

2. Make the create idempotent (also fine)

- name: Ensure visual-test label exists
  run: |
    gh label create "visual-test" \
      --description "Issues from automated visual regression tests" \
      --color "E06C75" \
      --force
    gh label create "run-visual-test" \
      --description "Run deterministic visual regression workflow on a PR" \
      --color "E06C75" \
      --force

3. Check a single label at a time (most surgical)

- name: Ensure visual-test label exists
  run: |
    for spec in 'visual-test|E06C75|Issues from automated visual regression tests' \
                'run-visual-test|E06C75|Run deterministic visual regression workflow on a PR'; do
      name="${spec%%|*}"
      rest="${spec#*|}"
      color="${rest%%|*}"
      desc="${rest#*|}"
      if gh label list --search "^${name}$" --json name --jq '.[].name' | grep -qx "${name}"; then
        echo "label ${name} already exists"
      else
        gh label create "${name}" --description "${desc}" --color "${color}"
      fi
    done

Related

• [Visual Test] Label preflight fails: gh label list only returns first 30 labels, run-visual-test is on page 2 #916 — first report of this exact preflight failure (runs Refactor: Split main.zig into modular game components #17, Refactor: Decompose App and Fix Vulkan Parity/SOLID issues #18). Still open.
• [Visual Test] PPM screenshot capture is unsupported #913, [Visual Test] Menu screenshot RMSE 0.0437 exceeds 0.015 tolerance after g_pipeline inherits alphaToCoverageEnable #911 — earlier engine-level visual-test failures that were diagnosed and fixed without touching this workflow file.

[github-actions]

Classification: Bug (CI/hotfix, not a feature request or question).

Validation: Confirmed against .github/workflows/visual-test.yml:55-68. The Ensure visual-test label exists step runs two unpaginated gh label list --json name --jq '.[].name' calls. gh label list defaults to 30 items per page (alphabetical), so on a repo with 41 labels any label sorting past position 30 (here, run-visual-test) is invisible to the guard, and gh label create then fails because the label is already present. Your run‑19 trace matches the run‑17 snippet in #916 byte‑for‑byte. The downstream screenshot, Lavapipe, and HomeScreen paths were never reached, so blaming engine code would be premature.

Duplicates / related:

• [Visual Test] Label preflight fails: gh label list only returns first 30 labels, run-visual-test is on page 2 #916 — identical failure, runs Refactor: Split main.zig into modular game components #17 / Refactor: Decompose App and Fix Vulkan Parity/SOLID issues #18, still open, no fix PR linked.
• [Visual Test] PPM screenshot capture is unsupported #913, [Visual Test] Menu screenshot RMSE 0.0437 exceeds 0.015 tolerance after g_pipeline inherits alphaToCoverageEnable #911 — earlier visual‑test failures that were attributed to engine code only because the preflight aborted before the build ever ran; they should be re‑evaluated once this gate is actually reachable.
• PR search for paginate / label list / visual-test against dev: no in‑flight fix branch. Nothing is currently racing this.

Scope: Correctly sized already. This is a 2‑line YAML edit in one workflow file; splitting it would only add ceremony. A single hotfix/visual-test-label-paginate PR targeting dev is the right shape.

Recommended next steps:

1. Open a hotfix/* PR against dev with Option 1 from the issue (gh label list --paginate ...) on both calls at visual-test.yml:57 and :62. Option 2 (--force) and Option 3 (--search) are also valid; Option 1 is the smallest delta that preserves the existing "skip if present" intent.
2. Once the preflight lands and the next scheduled run reaches step 10, watch for an actual engine‑level screenshot/diff result before declaring victory — that is the first time the gate will have executed end‑to‑end.
3. Close [Visual Test] Label preflight fails: gh label list only returns first 30 labels, run-visual-test is on page 2 #916 and this issue as duplicates of whichever lands first (recommend closing [Visual Test] Label preflight fails: gh label list only returns first 30 labels, run-visual-test is on page 2 #916 as a dup of this one, since this report is the more recent and more thoroughly cross‑referenced). Then re‑open / re‑investigate [Visual Test] PPM screenshot capture is unsupported #913 and [Visual Test] Menu screenshot RMSE 0.0437 exceeds 0.015 tolerance after g_pipeline inherits alphaToCoverageEnable #911 only if a real engine failure surfaces.

No missing reproduction data, no vague scope, nothing to ask the author for — the diagnosis is complete and the fix is mechanical.

opencode session  |  github run