test(profile): regenerate seed profile (FF150-primed, uBlock) + assert extension startup#1189
Open
vringar wants to merge 1 commit into
Open
test(profile): regenerate seed profile (FF150-primed, uBlock) + assert extension startup#1189vringar wants to merge 1 commit into
vringar wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR makes the seed Firefox profile fixture (test/profile.tar.gz) a meaningful cross-version compatibility check by priming it with Firefox N-1 and baking in a real extension (uBlock Origin), then strengthening restoration tests to assert the extension is actually active after load/recovery.
Changes:
- Add a regeneration script to reproducibly build the seed profile by installing uBlock Origin from AMO, warming history, pruning transient caches, and archiving.
- Strengthen profile restoration tests by adding a command that queries
AddonManagerto assert the baked-in extension is installed and active. - Update Firefox deployment to support privileged chrome-context querying and update release docs/workflow around regenerating the seed profile before bumps.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
test/test_profile.py |
Adds uBlock addon ID constant and a new command asserting addon installed+active via AddonManager; updates pref-read to use chrome context. |
scripts/regenerate-test-profile.py |
New out-of-band script to generate the seed profile fixture with a permanent uBlock install and required state. |
scripts/install-firefox.sh |
Updates the pinned Firefox tag to 151.0.4 release. |
openwpm/deploy_browsers/deploy_firefox.py |
Adds -remote-allow-system-access to enable privileged chrome-context operations. |
docs/Release-Checklist.md |
Documents the pre-bump step to regenerate and commit the seed profile fixture. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+54
to
+59
| # Allow WebDriver to access the parent process and privileged Gecko APIs | ||
| # (e.g. switching to the chrome context). Firefox 138+ gates this behind an | ||
| # explicit launch flag; without it commands such as reading preferences via | ||
| # the chrome context fail with "System access is required". | ||
| # https://firefox-source-docs.mozilla.org/remote/Security.html | ||
| fo.add_argument("-remote-allow-system-access") |
| return false; | ||
| }} | ||
| """) | ||
| self.logger.error(f"Got result: {result}") |
Comment on lines
+95
to
+98
| xpi_path = dest_dir / f"ublock_origin-{version}.xpi" | ||
| urllib.request.urlretrieve(xpi_url, xpi_path) | ||
| print(f"Saved XPI to {xpi_path} ({xpi_path.stat().st_size} bytes)") | ||
| return xpi_path |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1189 +/- ##
==========================================
- Coverage 62.11% 62.03% -0.09%
==========================================
Files 40 40
Lines 3907 3925 +18
==========================================
+ Hits 2427 2435 +8
- Misses 1480 1490 +10 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
vringar
force-pushed
the
test/profile-baseline-regen
branch
from
3c5343f to
e235988
Compare
…t extension startup
vringar
force-pushed
the
test/profile-baseline-regen
branch
from
e235988 to
01f62a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1188.
What
Make the seed profile fixture (
test/profile.tar.gz) a meaningful cross-version profile-compatibility check and reproducible, and strengthen the restoration tests to assert a baked-in extension actually starts up after load/recovery.This PR is based on the FF150 -> FF151 bump branch so CI exercises the real check (FF151 loading a FF150-primed profile). It rebases cleanly onto master once the bump lands.
N-1 priming rationale
The seed profile exists to confirm Firefox N can read a profile primed by the previous Firefox (N-1) — catching extension-signing / manifest / startup regressions that show up across version bumps. The committed fixture was written by Firefox 137.0.2 (~13 versions stale), so it tested nothing. The new baseline is primed with Firefox 150.0.2 (the previous stable), so the FF151 bump's CI exercises a genuine 150 -> 151 restoration.
The workflow is now documented in
docs/Release-Checklist.md: just before bumping the pin from N-1 to N, regenerate the fixture with Firefox N-1 and commit it.Changes
scripts/regenerate-test-profile.py(new) — Selenium script that launches Firefox (FIREFOX_BINARYfrom env), setstest_pref=True, installs uBlock Origin from AMO as a permanent addon (so it persists in the dump), warms up history, and writestest/profile.tar.gz. The signed XPI is resolved via the AMO v5 API (current_version.file.url) and installed withdriver.install_addon(path, temporary=False). It verifies uBlock is active via the privilegedAddonManager, then prunes transient caches (startupCache/cache2/safebrowsing/favicons) before archiving. The live AMO call runs only here, out-of-band — CI never touches addons.mozilla.org.Regenerated
test/profile.tar.gz— now Firefox 150.0.2-primed, contains uBlock Origin 1.71.0 (installed + active) andtest_pref=true.Strengthened
test/test_profile.py— newAssertExtensionActiveCommandqueries the liveAddonManagerfor the addon's state and asserts it is installed ANDisActive, proving startup (not just file presence). Wired intotest_seed_persistence(always) andtest_profile_recovery(whenseed_taris set). Existing pref/history assertions are kept. No live AMO call in the test path.Docs — script header documents the regeneration procedure;
docs/Release-Checklist.mdgains a pre-bump step.Verification
The strengthened tests pass on both Firefox versions, including the new uBlock-startup assertion:
test_seed_persistencetest_profile_recovery(with_seed_tar)