test(polluter): add polluter bisect script#16235
Conversation
MayaKirova
left a comment
There was a problem hiding this comment.
Maybe also just add a ReadMe with the info on how to use this (like the one in the PR description), so that we have it in the repo.
|
There has been no recent activity and this PR has been marked inactive. |
|
There has been no recent activity and this PR has been marked inactive. |
|
@IMinchev64 Is this ready for merge? |
There was a problem hiding this comment.
Pull request overview
Adds a developer-only “polluter bisect” utility to help isolate test files that pollute the environment and cause a chosen sentinel spec to fail when run in a suite.
Changes:
- Introduces a Node script that generates a deterministic runner spec and bisects the test set by repeatedly invoking
ng test. - Adds a README describing usage and modes (
before/all) plus a--skip-initialoption. - Wires the script into
package.jsonand ignores the generated runner spec in.gitignore.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| scripts/test-polluter-bisect/test-polluter-bisect.js | Implements the bisect logic, runner generation, and test execution via ng test. |
| scripts/test-polluter-bisect/README.md | Documents the purpose and usage of the bisect tool. |
| package.json | Adds the polluter:bisect npm script entry. |
| .gitignore | Ignores the generated polluter-runner.spec.ts file. |
|
@kdinev Generally yes, although @damyanpetev wanted to take a look before merging it just in case + now I see that the Copilot has left some comments which I would like to examine and address |
|
There has been no recent activity and this PR has been marked inactive. |
Description
Sometimes tests from the
igniteui-angularproject fail when run as part of the full suite, even though they pass when executed in isolation.The reason is that some tests pollute the testing environment (e.g. by leaving behind global state or async handles), which causes later tests to behave incorrectly.
With a suite of ~200 test files and more than 5000 tests, it’s not feasible to manually track down the culprit file.
This PR introduces a polluter-bisect script that streamlines the process of identifying polluting tests by:
before- only considers tests that run before the sentinel. (default)all- considers all tests in the suite and runs the sentinel last.This makes it possible to isolate polluting test files much faster than running the entire suite repeatedly.
The script is not intended to run in CI; it’s a developer tool to aid in diagnosing flaky tests.
Usage
From the root of the repo, run:
The script will iteratively run subsets of tests until it identifies the polluting test file that causes the sentinel to fail.
Additional information (check all that apply):
Checklist:
feature/README.MDupdates for the feature docsREADME.MDCHANGELOG.MDupdates for newly added functionalityng updatemigrations for the breaking changes (migrations guidelines)