Skip to content

fix(#7063): poll /wallet/balances/all with X-Admin-Key for large_tx events#7586

Open
Yzgaming005 wants to merge 1 commit into
Scottcjn:mainfrom
Yzgaming005:fix/7063-webhook-large-tx-balances-endpoint
Open

fix(#7063): poll /wallet/balances/all with X-Admin-Key for large_tx events#7586
Yzgaming005 wants to merge 1 commit into
Scottcjn:mainfrom
Yzgaming005:fix/7063-webhook-large-tx-balances-endpoint

Conversation

@Yzgaming005

Copy link
Copy Markdown
Contributor

Summary

The webhook dispatcher's large_tx detector was polling /api/balances,
which the live RustChain node no longer exposes — the unauthenticated
route was removed in favour of the admin-gated /wallet/balances/all
endpoint (X-Admin-Key header). As a result, the large_tx event was
silently dead and operators only saw 404s in the poller logs.

This change wires the poller up to the new endpoint with proper auth,
while staying backward-compatible with operators that have not (yet)
configured an admin key: the poll is skipped gracefully with a debug
log instead of hammering an unauthenticated surface that returns 503.

Changes

  • tools/webhooks/webhook_server.py (+74 / −13)
    • RustChainPoller gains admin_key and balances_path constructor
      parameters (both overridable for tests and future endpoint renames).
    • _get now forwards optional headers= so per-call auth can be
      injected without breaking the other poller call sites
      (/headers/tip, /api/stats, /api/miners).
    • _check_large_tx:
      • early-returns with a debug log when RUSTCHAIN_ADMIN_KEY is empty
        (so existing operators don't see noisy 503/401 on every cycle),
      • hits the new balances_path with X-Admin-Key: <key>,
      • accepts the canonical envelope
        {"balances": [{"miner_id", "amount_i64", "amount_rtc"}, ...], "total_i64", "total_rtc"},
      • falls back to the legacy bare-list response shape for operators
        on older nodes,
      • prefers amount_rtcamount_i64balanceamount for the
        numeric value, casting safely to float and skipping non-numeric
        rows instead of crashing the poller thread.
    • New env defaults + CLI flags: RUSTCHAIN_ADMIN_KEY
      --admin-key, RUSTCHAIN_BALANCES_PATH--balances-path
      (default /wallet/balances/all).
  • tools/webhooks/test_webhook_server.py (+207 / −3)
    • 11 new unit tests covering: admin-key gate, header injection,
      envelope vs. legacy response shape, threshold/credit/debit
      dispatch, malformed entries, unexpected shapes, and HTTP failure
      handling.

Why this approach

  • The existing poller already had clean separation between transport
    (_get) and detection (_check_large_tx); extending both with
    opt-in parameters keeps the change additive and lets the other
    detectors keep using unauthenticated endpoints unchanged.
  • The "skip when admin key is unset" branch is what makes this
    backward-compatible: an operator that has not configured
    RC_ADMIN_KEY on the node will see the same behaviour as before
    (no large_tx events, no noisy 503s in the logs) and can flip
    the new env var on at their own pace.
  • The dual-shape response handler covers both the live
    /wallet/balances/all contract (already used by
    rustchain_exporter, auto-pay.py, and the admin key test
    suite) and any older deployment that still returns a bare list, so
    rolling out a node upgrade isn't required to unblock operators on
    a stale build.
  • _get's new headers= parameter is a small, named change that
    future endpoints with non-empty auth headers can also use without
    re-introducing the same pattern.

Testing

  • pytest tools/webhooks/test_webhook_server.py -v → 15/15 pass
    (4 pre-existing + 11 new for #7063).
  • pytest tools/webhooks/ tests/test_webhook_admin_auth.py tests/test_webhook_server_helpers.py tests/test_webhook_client_helpers.py test_webhook_client.py
    51/51 pass, no regressions in the existing webhook surface.

Manual verification

  • Confirmed the new endpoint exists in node/rustchain_v2_integrated_v2.2.1_rip200.py
    (api_wallet_balances_all) and that its contract is
    {"balances": [{"miner_id", "amount_i64", "amount_rtc"}, ...]} with
    X-Admin-Key enforcement + 503 ADMIN_KEY_UNSET when RC_ADMIN_KEY
    is empty (see tests/test_wallet_transfer_admin_key_unset.py).
  • git log --oneline -1 on the branch:
    5edcd86b fix(#7063): poll /wallet/balances/all with X-Admin-Key for large_tx events.
  • Diff is +284 / −13 across 2 files; only webhook code + tests are
    touched.

Trade-offs

  • I did not change the deprecated node/rustchain_v2_active.py /
    deprecated/old_nodes/* copies — the dispatcher in tools/webhooks/
    always talks to whichever node URL the operator points it at
    (RUSTCHAIN_NODE / --node), so updating the dispatcher is
    sufficient; the legacy node files are out of scope for #7063.
  • The new env vars (RUSTCHAIN_ADMIN_KEY, RUSTCHAIN_BALANCES_PATH)
    follow the existing operator-config style in this file (see
    RUSTCHAIN_NODE, WEBHOOK_POLL_INTERVAL, LARGE_TX_THRESHOLD)
    rather than a separate config file. If maintainers prefer a
    config.yaml style, I'm happy to refactor in a follow-up.

Closes #7063

…rge_tx events

The webhook dispatcher's large_tx detector was polling /api/balances,
which the live node no longer exposes (the unauthenticated route was
removed in favour of the admin-gated /wallet/balances/all endpoint).
As a result, the large_tx event was silently dead and operators only
saw 404s in the poller logs.

Changes:
- Add RUSTCHAIN_ADMIN_KEY + RUSTCHAIN_BALANCES_PATH env defaults and
  --admin-key/--balances-path CLI flags so the dispatcher can authenticate.
- Plumb admin_key + balances_path into RustChainPoller; the existing
  balances_path default (/wallet/balances/all) is overridable so future
  endpoint renames are a config change, not a code change.
- Skip the poll gracefully when RUSTCHAIN_ADMIN_KEY is empty instead of
  hammering an unauthenticated surface that returns 503.
- Accept both the canonical envelope ({"balances": [...], "total_*": ...})
  and the legacy bare-list response, with miner_id/miner and
  amount_rtc/amount_i64/balance/amount field fallbacks.
- Extend _get to forward optional headers (X-Admin-Key) without breaking
  the other poller call sites.
- Add 11 focused unit tests covering: admin key gate, header injection,
  envelope vs. legacy response shape, threshold/credit/debit dispatch,
  malformed entries, unexpected shapes, and HTTP failure handling.
@github-actions

Copy link
Copy Markdown
Contributor

Welcome to RustChain! Thanks for your first pull request.

Before we review, please make sure:

  • Non-doc PRs have a BCOS-L1 or BCOS-L2 label
  • Doc-only PRs are exempt from BCOS tier labels when they only touch docs/**, *.md, or common image/PDF files
  • New code files include an SPDX license header
  • You've tested your changes against the live node

Bounty tiers: Micro (1-10 RTC) | Standard (20-50) | Major (75-100) | Critical (100-150)

A maintainer will review your PR soon. Thanks for contributing!

@github-actions github-actions Bot added BCOS-L1 Beacon Certified Open Source tier BCOS-L1 (required for non-doc PRs) size/L PR: 201-500 lines labels Jun 24, 2026
@Yzgaming005

Copy link
Copy Markdown
Contributor Author

Test Results & Manual Verification

$ pytest tools/webhooks/test_webhook_server.py -v
============================= 15 passed in 0.14s ==============================
$ pytest tools/webhooks/ tests/test_webhook_admin_auth.py \
        tests/test_webhook_server_helpers.py \
        tests/test_webhook_client_helpers.py test_webhook_client.py
============================= 51 passed in 2.39s ==============================

11 new tests cover: admin-key gate, header injection, envelope + legacy response, credit/debit dispatch, threshold skip, malformed entries, unexpected shapes, HTTP failure. All pre-existing webhook tests still pass (no regressions).

Key design decisions:

  • Made balances_path overridable (RUSTCHAIN_BALANCES_PATH, default /wallet/balances/all) so future endpoint renames are a config change, not a code change.
  • Made the admin-key gate opt-in (skip-when-empty) so existing operators without RC_ADMIN_KEY configured don't see noisy 503/401s on every poll cycle.
  • _get's new headers= parameter is additive — the other detectors (/headers/tip, /api/stats, /api/miners) keep their unauthenticated contract unchanged.

Looking forward to the review. 🙏

@jaxint jaxint left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Code reviewed - implementation verified. Changes align with project standards.

@Yzgaming005

Copy link
Copy Markdown
Contributor Author

⏸️ CI status note — the only red is test_fetchall_guard_passes_current_baseline (existing unannotated .fetchall() calls in main that aren't in the current baseline). This PR itself doesn't introduce any new fetchall calls — the failure is the shared infrastructure issue.

Unblocking: PR #7568 (chore(ci): refresh fetchall baseline for #7502) fixes the baseline and is ready for review. Once it lands, a rebase here will clear CI.

Will rebase this PR as soon as #7568 is merged. No action needed on the diff itself.

— Yzgaming005

@jaxint jaxint left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Code reviewed - implementation verified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BCOS-L1 Beacon Certified Open Source tier BCOS-L1 (required for non-doc PRs) size/L PR: 201-500 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Webhook large_tx polling uses stale balances endpoint

2 participants