test: fix flaky webhook E2E test#930
Closed
vdusek wants to merge 1 commit into
Closed
Conversation
The client Actor exited milliseconds after registering the ad-hoc webhook, so the platform could process the run-succeeded event before the webhook propagated, leaving the server Actor waiting until it timed out. Keep the client run alive briefly after add_webhook to close the race window.
vdusek
added
adhoc
github-actions
Bot
added
the
tested
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #930 +/- ##
=======================================
Coverage 86.94% 86.94%
=======================================
Files 48 48
Lines 2942 2942
=======================================
Hits 2558 2558
Misses 384 384
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
Author
|
Superseded by #920, which includes this fix as part of consolidating the test polling helpers. |
vdusek
added a commit
that referenced
this pull request
Jun 5, 2026
…flake via polling (#920) ## Summary The SDK counterpart of apify/apify-client-python#844: introduce a single shared `tests/_utils.py` with one polling helper, and migrate the whole test suite to it — replacing fixed sleeps and hand-rolled retry loops that caused flakiness. ## Changes - **Shared `tests/_utils.py`.** One `poll_until_condition(fn, condition=bool, *, timeout, poll_interval, backoff_factor)` helper — identical to the one in apify-client-python#844 — polls a sync-or-async callable until a condition holds or a wall-clock timeout expires. `backoff_factor=2` subsumes the former `call_with_exp_backoff`; `timeout=0` is the "call once" case. The `maybe_await` adapter, `generate_unique_resource_name`, and the shared RSA crypto test keys move here too. The per-package `tests/integration/_utils.py` and `tests/e2e/_utils.py` are removed, and cross-test imports (e.g. the crypto keys, previously imported from the `test_crypto` module) now go through this single module. - **Request queue tests.** ~50 polling call sites in `test_request_queue.py` migrated to `poll_until_condition`. The single/shared timeout is centralized in an `rq_poll_timeout` fixture (`timeout=0` single, `timeout=30` shared), backed by an `rq_access_mode` fixture — replacing the access-mode derivation block previously copy-pasted into every test. The shared-mode request-ordering relaxations from #931 are preserved on top. - **Deflaked tests.** Fixed the flaky `test_actor_adds_webhook_and_receives_event` e2e test (the client now stays alive 5 s after `add_webhook`, and the unbounded `INITIALIZED` startup loop becomes bounded polling) — replaces #930. Replaced the `retry_counter` loops in `test_actor_charge.py` (4×) and fixed sleeps in `test_actor_lifecycle`, `test_actor_key_value_store`, and `test_apify_event_manager` with condition polling. Fixed sleeps that are semantically required are intentionally kept: negative checks ("event must NOT fire"), mtime-granularity checks, simulated latency, and sleeps inside deployed Actor `main` bodies (where the helper is unavailable). Verified: lint, type-check, and the full unit suite pass; integration/e2e require a platform token and run in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
test_actor_adds_webhook_and_receives_eventoccasionally fails with the server Actor endingTIMED-OUTinstead ofSUCCEEDED(e.g. this CI run).ACTOR_RUN_SUCCEEDEDwebhook, so the platform can process the run-succeeded event before the webhook has propagated — the webhook then never fires and the server Actor blocks inhandle_request()until its 600 s run timeout.Actor.add_webhook()to close the race window.