feat(webhooks): verifyAndParse* API for compressed payloads (CHA-3071)#391
Conversation
…HA-3071) Co-authored-by: Cursor <cursoragent@cursor.com>
Replaces VerifyAndDecodeWebhook / DecompressWebhookBody with the cross-SDK contract documented at https://getstream.io/chat/docs/node/webhooks_overview/. Package-level helpers in stream_chat: Primitives: UngzipPayload - gzip magic-byte detection + inflate DecodeSqsPayload - base64 then ungzip-if-magic DecodeSnsPayload - alias for DecodeSqsPayload VerifySignature - constant-time HMAC-SHA256 comparison ParseEvent - JSON -> *Event (typed, with ExtraData fallback for unknown event types) Composite (return *Event): VerifyAndParseWebhook VerifyAndParseSqs VerifyAndParseSns The composite functions auto-detect compression from body bytes, so the same handler stays correct whether or not Stream is currently compressing payloads, and behind middleware that auto-decompresses. Client instance methods (Client.VerifyAndParse*) mirror the three composite helpers and pull the API secret from the receiver. The legacy Client.VerifyWebhook(body, signature) bool helper is unchanged for backward compatibility. ErrInvalidWebhookSignature is preserved as the sentinel error for HMAC mismatch. Co-authored-by: Cursor <cursoragent@cursor.com>
RFC 1952 defines the gzip magic number as the two-byte sequence 1F 8B; the third byte (CM) is informational and not part of the identifier. Trim the magic check from three bytes to two to match the spec and stay consistent with the reference implementations in the public docs. Co-authored-by: Cursor <cursoragent@cursor.com>
The compression section referenced VerifyAndDecodeWebhook, DecompressWebhookBody and a payloadEncoding argument that do not exist in this SDK. Rewrite to document the actual public surface: VerifyAndParseWebhook / VerifyAndParseSqs / VerifyAndParseSns plus the lower-level UngzipPayload, VerifySignature and ParseEvent helpers. CHA-3071 Co-authored-by: Cursor <cursoragent@cursor.com>
io.ReadAll followed by a deferred Close swallowed the trailing CRC and length-trailer error that compress/gzip only reports when the reader is closed. Replace the defer with explicit Close handling so a corrupt or truncated gzip stream is reported as a gzip error instead of falling through to a misleading invalid-signature failure. CHA-3071 Co-authored-by: Cursor <cursoragent@cursor.com>
mogita
left a comment
There was a problem hiding this comment.
Cross-SDK review pass for CHA-3071. Two inline comments — see below.
…y verify
decode_sns_payload now JSON-parses the SNS HTTP notification envelope
({"Type":"Notification","Message":"..."}) and extracts the inner
Message field before running the SQS pipeline. Falls through to the
pre-extracted Message string when the input is not a JSON envelope so
existing call sites keep working.
Legacy Client.VerifyWebhook now uses hmac.Equal for constant-time
comparison instead of bytes.Equal, closing the same timing leak we
already closed in WebhookHelpers.VerifySignature.
Test fixture adds a realistic SNS HTTP notification body that all
SDKs in this rollout can share.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
Cross-SDK coordination: unifying webhook exception types After the review pass across all 6 SDKs in this rollout and team discussion, we're consolidating the new webhook exception strategy to a single unified error type rather than the split (signature vs parse sentinels) being introduced in this PR. The Webhook Handling Spec on Notion (CHA-2961) has been revised to reflect this — §5.2 / §5.3 / §7 now specify a single class. Why unified: From a customer's perspective, all failure modes — signature mismatch, gzip decompression failure, base64 decode failure, SNS envelope failure, JSON parse failure, missing schema field — terminate at the same error-handling branch in customer code. A signature/parse split adds structural complexity without changing customer behavior. Customers who want to filter security logs for signature mismatches specifically can do so via error message text or Sentinel name: Per-SDK naming across the rollout:
Asks for this PR:
This same comment is being posted on all 6 SDK PRs (JS / Go / Ruby / PHP / Java / .NET) for coordination. Happy to discuss naming or scope tradeoffs. |
…n fixtures (CHA-3071) Per Tommaso's suggestion, align the gzip helper with the GNU `gunzip` command name. The function was added in this PR and not yet released, so this is a straight rename with no back-compat alias. Adds Tommaso's reference fixtures to the test suite as named cases so future SDKs can sanity-check against the same payloads: aGVsbG93b3JsZA== -> helloworld (base64) H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA -> helloworld (base64+gzip) Co-authored-by: Cursor <cursoragent@cursor.com>
…-3071) Per cross-SDK coordination (mogita's review across the 6 sibling SDK PRs), every webhook failure path now wraps a single sentinel error. Callers can do errors.Is(err, stream.ErrInvalidWebhook) for a unified check, or strings.Contains on the message for mode-specific filtering. Replaces the previously-unreleased ErrSignatureMismatch with ErrInvalidWebhook and threads it through every primitive: VerifySignature -> 'signature mismatch: ...' GunzipPayload -> 'gzip decompression failed: ...' DecodeSqsPayload -> 'invalid base64 encoding: ...' ParseEvent -> 'invalid JSON payload: ...' The legacy Client.VerifyWebhook (returning bool) is unchanged - still uses hmac.Equal for constant-time comparison. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Shipped in
Docs side, docs-content#1276 now carries the per-SDK error-class table on the Webhooks overview page. |
…-3071) Stream does not ship an X-Signature on SQS or SNS deliveries - those transports ride AWS-internal infrastructure (IAM-authenticated queues and AWS-signed SNS notifications), so HMAC verification on top is theatre. The package helpers now treat empty-string signature + secret as a skip-verification signal, and the instance methods take signature as a variadic argument. - VerifyAndParseSqs(body, '', '') -> decode + parse - VerifyAndParseSqs(body, sig, secret) -> decode + verify + parse - client.VerifyAndParseSqs(body) -> decode + parse - client.VerifyAndParseSqs(body, sig) -> + verify with client secret Passing only one of (signature, secret) returns an error wrapping ErrInvalidWebhook. The HTTP-webhook path is unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
…ndParseWebhook; docs + tests Co-authored-by: Cursor <cursoragent@cursor.com>
…bhookSignatureError + parse_*; Python WebhookSignatureError; guard test init without STREAM_*
Summary
Adds first-class support for gzip-compressed webhook payloads (HTTP webhooks, SQS, SNS) and exposes a stable
VerifyAndParse*API that mirrors the cross-SDK contract published in Webhooks Overview.New public API
Primitives (in
webhook.go, intended to be composable):GunzipPayload(body) ([]byte, error)— gzip-magic-byte detection, no-op when not compressedDecodeSqsPayload(body) ([]byte, error)— base64 decode then gunzip-if-magicDecodeSnsPayload(notificationBody) ([]byte, error)— JSON-parse the SNS HTTP notification envelope, extract the innerMessage, then run the SQS pipeline. Falls through to a pre-extractedMessagestring when the input is not a JSON envelopeVerifySignature(body, signature, secret) error— HMAC-SHA256 over the uncompressed body, with a constant-time comparison (matters for the HTTP webhook path where theX-Signatureheader is exposed publicly; SQS / SNS deliveries arrive over AWS-internal transports where timing-attack resistance is not strictly required). Returns a non-nil error wrappingstream.ErrInvalidWebhookon mismatch — see Unified error handling below.ParseEvent(payload) (*Event, error)— JSON → typed*EventComposites (return a typed
*Event):VerifyAndParseWebhook(body, signature, secret) (*Event, error)VerifyAndParseSqs(body, signature, secret) (*Event, error)VerifyAndParseSns(body, signature, secret) (*Event, error)Client-bound versions on
*Clientuse the configured app secret automatically.Backwards compatibility
Client.VerifyWebhookis preserved and delegates toVerifySignature. Existing callers continue to work unchanged for plain (uncompressed) bodies.Unified error handling
Per cross-SDK coordination (mogita's review across the 6 sibling SDK PRs), every webhook failure path now terminates at a single sentinel error —
stream.ErrInvalidWebhookso customers only need one error-handling arm. The message text identifies which failure mode fired so callers that want to differentiate (security logging, retry policy) can filter on substring:signature mismatchinvalid base64 encodinggzip decompression failedinvalid JSON payloadUse
errors.Is(err, stream.ErrInvalidWebhook)for the unified check; the inner cause (when present) is preserved via%wwrapping.VerifySignaturenow returnserror(wasbool) so it surfaces the mode-specific message; the legacyClient.VerifyWebhook(bool return, constant-timehmac.Equal) is untouched.Tests
webhook_test.gocovers plain / gzip / base64 / base64+gzip payloads, signature mismatches, malformed bytes, JSON parsing into*Event, and a backwards-compat path forClient.VerifyWebhook. Linked Linear ticket: CHA-3071.Golden test fixtures (Tommaso)
Added shared reference fixtures to the test suite so future SDKs can sanity-check decoders against the same payloads:
Test plan
go vet ./...cleango test -c .(test binary compiles clean)go test ./...runs in CI (packageinit()requiresSTREAM_KEY/STREAM_SECRET)