Skip to content

test: fix flaky test-runner coverage threshold test#64051

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
trivikr:test-runner-run-coverage
Jun 23, 2026
Merged

test: fix flaky test-runner coverage threshold test#64051
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
trivikr:test-runner-run-coverage

Conversation

@trivikr

@trivikr trivikr commented Jun 21, 2026

Copy link
Copy Markdown
Member

This fixes a flaky assertion in parallel/test-runner-run-coverage.mjs.

The threshold subtest was asserting that process.exitCode was not already
1 before starting its coverage run. That value is process-global, so it
can be affected by other run() coverage activity in the same process.

The test now clears process.exitCode before the threshold check and
restores the original value with a cleanup hook.

Refs: https://github.com/nodejs/reliability/issues?q=%22test-runner-run-coverage%22

Example
not ok 3959 parallel/test-runner-run-coverage
  ---
  duration_ms: 710.47400
  severity: fail
  exitcode: 1
  stack: |-
    Test failure: 'should run with coverage and fail when below line threshold'
    Location: test/parallel/test-runner-run-coverage.mjs:157:11
    AssertionError [ERR_ASSERTION]: Expected "actual" to be strictly unequal to: 1
        at TestContext.<anonymous> (file:///home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-runner-run-coverage.mjs:160:14)
        at Test.runInAsyncScope (node:async_hooks:226:14)
        at Test.run (node:internal/test_runner/test:1382:25)
        at Suite.processPendingSubtests (node:internal/test_runner/test:960:18)
        at Test.postRun (node:internal/test_runner/test:1522:19)
        at Test.run (node:internal/test_runner/test:1447:12)
        at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
        at async Suite.processPendingSubtests (node:internal/test_runner/test:960:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: 1,
      expected: 1,
      operator: 'notStrictEqual',
      diff: 'simple'
    }

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Jun 21, 2026
@atlowChemi

Copy link
Copy Markdown
Member

@trivikr I think the commit should be under the test: submodule

Reset process.exitCode before running the coverage threshold check.
This lets the test verify the exit code set by that run() call without
depending on process-global state from earlier coverage runs.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
@trivikr trivikr force-pushed the test-runner-run-coverage branch from 1d1026a to d73ab26 Compare June 22, 2026 06:31
@trivikr trivikr changed the title test_runner: fix flaky run coverage threshold test test: fix flaky test-runner coverage threshold test Jun 22, 2026
@trivikr

trivikr commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

I think the commit should be under the test: submodule

Yes. Since the code changes are only in test files.
I've updated the commit message and requested a review.

@trivikr trivikr requested a review from atlowChemi June 22, 2026 06:32
@trivikr trivikr added flaky-test Issues and PRs related to the tests with unstable failures on the CI. request-ci Add this label to start a Jenkins CI on a PR. labels Jun 22, 2026
@trivikr trivikr added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jun 22, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 22, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@trivikr trivikr added commit-queue Add this label to land a pull request using GitHub Actions. fast-track PRs that do not need to wait for 72 hours to land. labels Jun 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Fast-track has been requested by @trivikr. Please 👍 to approve.

@nodejs-github-bot nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Jun 22, 2026
@nodejs-github-bot

This comment was marked as outdated.

@trivikr trivikr removed the commit-queue-failed An error occurred while landing this pull request using GitHub Actions. label Jun 22, 2026
@JakobJingleheimer JakobJingleheimer added commit-queue Add this label to land a pull request using GitHub Actions. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. labels Jun 23, 2026
@trivikr trivikr removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 23, 2026
@trivikr trivikr removed the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Jun 23, 2026
@trivikr

trivikr commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

@JakobJingleheimer I removed commit-queue labels as they need two 👍 for fast-track comment.

@atlowChemi atlowChemi added the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 23, 2026
@nodejs-github-bot nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Jun 23, 2026
@nodejs-github-bot

This comment was marked as outdated.

@trivikr

trivikr commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

The commit-queue failed with error

✖  7:120    Trailer should be <= 120 columns.         line-length

Because the Refs in PR description was 133 lines long

Refs: https://github.com/nodejs/reliability/issues?q=sort%3Aupdated-desc%20is%3Aissue%20state%3Aopen%20%22test-runner-run-coverage%22

I've changed it to

Refs: https://github.com/nodejs/reliability/issues?q=%22test-runner-run-coverage%22

And have requested commit-queue again

@trivikr trivikr added commit-queue Add this label to land a pull request using GitHub Actions. and removed commit-queue-failed An error occurred while landing this pull request using GitHub Actions. labels Jun 23, 2026
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 23, 2026
@nodejs-github-bot nodejs-github-bot merged commit 1e7c852 into nodejs:main Jun 23, 2026
104 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 1e7c852

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. fast-track PRs that do not need to wait for 72 hours to land. flaky-test Issues and PRs related to the tests with unstable failures on the CI. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants