Skip to content

feat: PostgreSQL-backed dataset search with filters, sync pipeline, and file download#127

Merged
elainefan331 merged 29 commits into
stagingfrom
dev-fan
May 28, 2026
Merged

feat: PostgreSQL-backed dataset search with filters, sync pipeline, and file download#127
elainefan331 merged 29 commits into
stagingfrom
dev-fan

Conversation

@elainefan331
Copy link
Copy Markdown
Collaborator

Description:
Overview
Replaces the legacy CGI-based search with a full PostgreSQL implementation, introduces a rich filter UI, and adds file download capabilities for search results.

Backend & Infrastructure

  • Added search tables migration and updated DB config for local Postgres
  • Replaced CGI search with direct PostgreSQL queries in searchAllDatabases
  • Added incremental sync script to populate ioviews and iolinks from CouchDB, refactored with transactions, concurrency, and error handling
  • Sync now pulls database list from registry

Search Filters

  • Modality-aware combobox for Data type filter
  • Draggable age-range slider with per-handle clearing
  • Multi-select File types filter (dataset-level) backed by iolinks table
  • Dataset-level modality filter with AND/OR mode
  • Placeholder hints on task/session/run keyword fields
  • Min/max count fields paired on one row
  • Run count shown in subject card

Resolves #82

  • Added incremental sync script to populate ioviews and iolinks from CouchDB
  • Sync now pulls database list from registry

Resolves #83

  • Replaced CGI search with direct PostgreSQL queries in searchAllDatabases
  • Added file type filter to search using iolinks table
  • Added file download endpoints for dataset search results
  • Mac/Linux and Windows script options for downloading filtered files

@elainefan331 elainefan331 merged commit 6171e7b into staging May 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add file-level search in the search backend Update and syncrhonize the cross-database search SQL database

1 participant