Skip to content

Stop user-data errors leaking to Sentry from migrations#177

Merged
abnegate merged 9 commits into
mainfrom
fix-migration-sentry-leak
May 7, 2026
Merged

Stop user-data errors leaking to Sentry from migrations#177
abnegate merged 9 commits into
mainfrom
fix-migration-sentry-leak

Conversation

@premtsd-code
Copy link
Copy Markdown
Contributor

Summary

Per-resource user errors thrown by Destinations/Appwrite.php were leaking to Sentry through the migrations worker. This PR fixes the routing at the library layer.

  • Convert ~15 throw new Exception(...) user-error sites in create*() methods to setStatus + addError + return false. No exception bubbles for expected outcomes; the failure is recorded where detected.
  • Per-resource wrap at import() becomes a pure safety net: record + rethrow. Library bugs and infra failures bubble through; user errors stay in the migration report.
  • Row-flush block catches DuplicateException and StructureException from the database adapter and re-throws as Migration\Exception so the wrap classifies them as user-facing.
  • Critical context: Migration\Exception is now the library's marker for "user-facing migration error". The Appwrite worker uses instanceof Migration\Exception to decide Sentry routing.

Motivation

Errors like "Document already exists", "Attribute already exists", "Cannot set default value for array column", "Invalid document structure: Missing required attribute X" were being thrown with code 0 or PDO codes (e.g. 23000), and the migration worker's code === 0 || >= 500 heuristic re-published every one of them to Sentry. ~80% of the noise on the appwrite-queue-migrations Sentry stream came from this path.

After this PR, expected user-data conditions never throw; only bugs/infra do.

Test plan

  • Migration with a duplicate column → "Attribute already exists" appears in the migration report, no Sentry event
  • Migration with a duplicate row → "Document already exists" in the report only
  • Migration with malformed source data → "Invalid document structure" in the report only
  • Library bug (synthetic TypeError) → recorded in report and reaches Sentry with full trace

Two changes in Destinations/Appwrite.php:

1. Per-resource wrap rethrows non-Migration\Exception throwables. User
   errors still record on the migration document and the loop continues
   (existing behaviour). Library bugs and infra failures now bubble to
   the worker boundary so they can be routed to Sentry instead of being
   silently absorbed.

2. Row-flush match block catches DuplicateException and StructureException
   from the database adapter and rethrows them as Migration\Exception so
   the per-resource wrap above recognises them as user-facing failures.
   This keeps "Document already exists" / invalid structure messages in
   the migration report only.
Convert per-resource user-error throws into setStatus + addError + return false
in the create*() methods (createDatabase, createEntity, createField,
createIndex, createRecord). Failures are recorded for the migration report
where they're detected, and the import loop continues to the next resource —
matching the existing skip-from-cache pattern at line 1015 and the
documents-DB skip at line 535.

The per-resource wrap at import() simplifies to a pure safety net: record
for the user, then rethrow. By construction, anything reaching it is a
library bug or infra failure, which the worker boundary routes to Sentry.

Row-flush DuplicateException / StructureException are caught at the flush
site and recorded inline — same shape as the rest of the create*() methods.

Behaviour preserved: user errors continue to appear in the migration report,
the loop continues across resources, the migration ends with status=failed
when any resource failed. Only routing to Sentry changed.
DB adapters already normalise DuplicateException's message to
'Document already exists', so a multi-type catch with $e->getMessage()
collapses both branches into one without changing user-visible behaviour.

Trimmed the safety-net comment to a single line.
Same shape as the row-flush combine: multi-type catch with a small
ternary for the per-type message. Two pairs collapsed: the single
attribute path and the relationship two-way path (which differ only
in 'Attribute limit exceeded' vs 'Column limit exceeded').
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 5, 2026

Greptile Summary

This PR converts ~15 throw new Exception(...) user-error sites in Destinations/Appwrite.php to setStatus + addError + return false, preventing expected migration failures (duplicate attributes, missing tables, invalid IDs, etc.) from ever throwing and leaking to Sentry. A new inner try/catch in the batch-flush block of createRecord() traps DuplicateException and StructureException from the database adapter with the same non-throwing pattern.

  • User-facing errors in all create*() methods are now recorded in-place and return false, so they never reach import()'s catch block or the worker's Sentry routing logic.
  • createField() and createIndex() infra-failure catch blocks now re-throw the raw \\Throwable instead of wrapping it as Migration\\Exception('Failed to create column/index'), preserving the original exception type and message in the migration report.
  • import() itself was not modified; its catch (\\Throwable) still records errors without re-throwing, meaning all exceptions — user and infra alike — are ultimately absorbed there.

Confidence Score: 4/5

Safe to merge for the primary goal of silencing user-data errors; the claim that infrastructure failures now reach Sentry is not backed by the code.

The user-error suppression changes are consistent and correct throughout all create*() methods. The one gap is that import()'s catch block was not changed — it still swallows every exception without rethrowing — so the raw Throwable re-throws from createField() and createIndex() never escape to the worker's Sentry router. The PR achieves its noise-reduction goal, but the stated secondary goal of routing genuine infra failures to Sentry is not implemented.

src/Migration/Destinations/Appwrite.php — specifically the unchanged import() catch block (lines 402-417) and whether it should selectively rethrow non-user exceptions.

Important Files Changed

Filename Overview
src/Migration/Destinations/Appwrite.php ~15 user-error throw sites converted to setStatus+addError+return false; createField/createIndex infra-failure catches now rethrow raw exceptions rather than wrapping as Migration\Exception; batch-flush block gains DuplicateException/StructureException catches; import() itself was not changed.

Reviews (5): Last reviewed commit: "Fix message: 'Column' -> 'Attribute' lim..." | Re-trigger Greptile

Comment thread src/Migration/Destinations/Appwrite.php Outdated
Comment thread src/Migration/Destinations/Appwrite.php
The catch (\Throwable) cleanup blocks in createField and createIndex
were rewrapping any thrown exception as a Migration\Exception with
a generic 'Failed to create column/index' message. With the worker's
instanceof MigrationException gate, this silenced real library bugs
(TypeError, RuntimeException, PDOException) from Sentry — the exact
opposite of what the surrounding PR aims to do.

Rethrow the original cause instead, so the safety net at import()
records the real message for the user and the worker boundary routes
non-MigrationException throwables to Sentry with full trace.
…leak

# Conflicts:
#	src/Migration/Destinations/Appwrite.php
Comment thread src/Migration/Destinations/Appwrite.php
The earlier 'throw $e' aborted the batch on the first failure, leaving
items #6-N at STATUS_PENDING and skipping the rest of the source's
fetch loop. Restore the original 'absorb-and-continue' shape so every
resource in the batch is attempted and recorded — failures all land
in the migration report and the user retries the run cleanly.

Trade-off: library bugs (TypeError, etc.) caught here are absorbed and
don't reach Sentry. Acceptable — the deterministic-bug case fails the
same way for every resource, so absorbing produces a uniform 'all
errored' report instead of '1 errored + N pending'. User errors are
still kept out of Sentry by the worker's instanceof MigrationException
gate, which is the primary goal of this PR.
@abnegate abnegate merged commit 759d6d6 into main May 7, 2026
4 checks passed
@abnegate abnegate deleted the fix-migration-sentry-leak branch May 7, 2026 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants