Skip to content

Add cloud storage readiness diagnostics#5

Open
andrei-hasna wants to merge 1 commit into
mainfrom
openloops/open-conversations/96413f58-d4c6-4d72-9578-775b7b827606-ebee490b
Open

Add cloud storage readiness diagnostics#5
andrei-hasna wants to merge 1 commit into
mainfrom
openloops/open-conversations/96413f58-d4c6-4d72-9578-775b7b827606-ebee490b

Conversation

@andrei-hasna

Copy link
Copy Markdown
Contributor

Summary

  • add explicit storage readiness diagnostics for local SQLite/files vs remote Postgres/AWS behavior without exposing database URLs
  • add opt-in cloud-runtime table grouping for messages/read-state while keeping default storage sync metadata-focused
  • add message UUID/read-state sync safeguards, duplicate UUID preflight diagnostics, composite-key conflict logging, and MCP/CLI coverage
  • document privacy, attachment, migration, and production AWS approval gates

Validation

  • HOME=/tmp/open-conversations-test-home bun test src/lib/storage-sync.test.ts src/lib/pg-migrations.test.ts src/cli/storage-contract.test.ts src/mcp/tools/storage-contract.test.ts src/mcp/tools/storage.test.ts src/lib/messages.test.ts src/lib/channels.test.ts src/lib/channel-notifications.test.ts src/lib/sessions.test.ts src/lib/summary.test.ts src/mcp/tools/messaging.test.ts src/mcp/tools/channels.test.ts -> 304 pass, 0 fail
  • HOME=/tmp/open-conversations-test-home bun test src/lib/db.test.ts src/lib/storage-sync.test.ts src/cli/storage-contract.test.ts src/mcp/tools/storage.test.ts -> 44 pass, 0 fail
  • HOME=/tmp/open-conversations-test-home bun test -> final rerun 966 pass, 0 fail
  • bun run typecheck
  • HOME=/tmp/open-conversations-build-home TMPDIR=/tmp TEMP=/tmp TMP=/tmp XDG_RUNTIME_DIR=/tmp/bun-runtime XDG_CACHE_HOME=/tmp/bun-cache bun run build
  • storage status/readiness JSON grep checks did not print postgres:// DSNs
  • staged and branch-diff credential scans clean

Notes

  • no live Postgres/RDS/S3 smoke was run; production mutation, migrations, secrets, and AWS changes remain gated for explicit approval
  • lefthook is not installed in PATH in this worker environment, so hook execution was unavailable during commit/push

@andrei-hasna

Copy link
Copy Markdown
Contributor Author

Merge review update: blocking for follow-up fixes before this PR can be safely merged.

Validation in the isolated OpenLoops worktree at head c278bdc:

  • Focused storage/db/CLI/MCP suite passed: 55 pass, 0 fail.
  • Full suite passed with writable local HOME: 966 pass, 0 fail.
  • bun run typecheck passed.
  • bun run build passed.

Blocking findings:

  1. Duplicate local message UUIDs remain diagnosable in SQLite, but cloud-runtime push selects all UUID-bearing messages and upserts remote rows on uuid. That can silently collapse duplicate local messages instead of enforcing the readiness gate. See src/lib/db.ts:846, src/lib/storage-sync.ts:538, and src/lib/storage-sync.ts:943.
  2. The PostgreSQL migration backfills blank UUIDs and immediately creates a unique message UUID index. Existing remote rows with duplicate non-empty UUIDs can make automatic migrations fail before push/pull; the current tests use a SQLite-backed fake remote and do not prove real Postgres duplicate behavior. See src/lib/pg-migrations.ts:88 and src/lib/storage-sync.ts:378.
  3. CLI/MCP sync paths collect per-table errors but can still present the operation as successful, and MCP storage sync drops detailed table errors from the response. See src/cli/storage.ts:25 and src/mcp/tools/storage.ts:162.
  4. GitHub merge policy evidence is incomplete: PR is clean/mergeable, but there are no submitted reviews and gh pr checks reports no checks on the branch.

I created follow-up todos task 28f03e62 to fix these PR blockers. No merge was attempted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant