Skip to content

App Catalog #389

Open
krokicki wants to merge 31 commits into
mainfrom
app-catalog
Open

App Catalog #389
krokicki wants to merge 31 commits into
mainfrom
app-catalog

Conversation

@krokicki

Copy link
Copy Markdown
Member

This PR adds an App Catalog that lets users publish ("share") apps they've added to a shared, server-wide catalog, and lets other users add those listings to their own app collection. It also reworks the Apps area into a tabbed layout, moves tool requirement verification from submit-time (server) to job runtime (compute node), and removes the confusing manifest version field.

What changed

Backend — catalog

  • New table app_listings (fileglancer/database.py, migration d9f1a3c5e208_add_app_listings_table.py): owner_username, url, manifest_path, branch, name, description, published_at, updated_at, with a unique constraint on (owner_username, url, manifest_path) and an index on owner_username.
  • CRUD helpers in database.py: list_app_listings, get_app_listing, get_app_listings_by_owner, get_app_listing_for_app, create_app_listing (rejects duplicates), update_app_listing (owner-scoped, editable name/description), delete_app_listing (owner-scoped).
  • New models in model.py: AppListing, ShareAppRequest, UpdateAppListingRequest, plus validate_catalog_listing_name / resolve_catalog_listing_name (non-empty, trimmed). UserApp gains an optional listing_id so the UI knows whether the user has already shared it.
  • New endpoints in server.py:
    • GET /api/catalog — list all listings (newest first)
    • POST /api/catalog — share one of the caller's own apps (404 if not owned, 409 on duplicate)
    • PATCH /api/catalog/{id} — edit name/description of a listing you own
    • DELETE /api/catalog/{id} — unshare a listing you own
    • POST /api/catalog/{id}/add — re-fetch the manifest/branch and upsert the listing's app into the caller's own apps (409 if already added)
    • GET /api/user/apps now annotates each app with its listing_id.

Backend — requirements at runtime instead of submit-time

  • submit_job no longer calls verify_requirements (server PATH). Instead, build_requirements_check() (apps/core.py) emits a bash snippet injected into the job script after PATH/conda/env setup but before pre_run/command. On any unmet requirement it prints all errors to stderr and exit 1s, failing the job with a readable message.
  • Requirement parsing tightened to reject compound/chained/comma specs (pixi>=0.40,<0.60), shared via _validate_requirements in model.py (used by both AppManifest and AppEntryPoint).

Removed: manifest version field

  • Dropped from AppManifest (model.py), the pixi/nextflow adapters, the TS AppManifest type, AppInfoDialog, AppLaunchForm, and AuthoringApps.md.

Frontend — tabbed Apps area

  • New AppsLayout with tabs My Apps / App Catalog / Jobs (job-count badge), nested routes under /apps (App.tsx). Navbar collapses the separate "Jobs" item into "Apps".
  • New Catalog page with search (name/description/sharer) and a "hide already installed" filter; ListingCard + ListingInfoDialog.
  • New DeleteAppDialog confirmation before removing an app.
  • AppInfoDialog gains an inline "Share to Catalog" form (keeps the dialog open through sharing) and Share/Unshare actions; AppCard shows a "Shared" badge.
  • New TanStack Query hooks in appsQueries.ts: useCatalogQuery, useShareAppMutation, useUpdateListingMutation, useUnshareListingMutation, useAddFromListingMutation.
  • FgButton loading state now overlays the spinner without changing button size; Spinner accepts sizeClasses and renders no text node when text is unset.

Tests

  • tests/test_apps.py: build_requirements_check coverage (present/missing/version compare/compound-reject/pipefail).
  • tests/test_catalog_endpoints.py: new endpoint coverage (~417 lines).

Incidental fixes to pre-existing issues

These were not introduced by the catalog work — they pre-dated it or arrived via other changes bundled on the branch — and were cleaned up along the way:

  • _find_manifests_in_repo raised on the first adapter failure (apps/core.py), so one adapter's error could mask a later adapter that would have handled the repo. This behavior came from the ticket-386 "multiple values" change (commit 1592067c), not the catalog work. Fixed to collect all adapter errors, return as soon as any adapter succeeds, and raise an aggregated error only when no adapter produced a manifest. Tests added.
  • merge_requirements matched the same requirement regex up to three times per entry (apps/core.py). Pre-existing on main; tidied to match once while consolidating the requirement-spec regex.

@StephanPreibisch @JaneliaSciComp/fileglancer

krokicki and others added 30 commits June 5, 2026 11:14
Users can publish their apps to a catalog browsed by everyone, and add
listings to their own collection. Adding from the catalog creates an
independent UserApp; the listing stores only metadata (no manifest
snapshot) so there is no staleness or drift to manage. The Apps page's
new "Browse Catalog" and "Add from URL" buttons replace the old single
"Add App" entry point.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trash icon on the app card and Delete button in the info dialog both
open a confirmation dialog before calling the remove mutation, matching
the Stop Service confirmation pattern in JobDetail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the standalone /catalog route and the separate Catalog/Jobs
navbar entries with a tab bar on /apps that switches between My Apps,
App Catalog, and Jobs. The Apps navbar entry now carries the active-job
badge that used to live on the Jobs link.

Tabs are a small NavLink-based component (not Material Tailwind's Tabs)
since the latter is built around in-memory Tabs.Panel content rather
than URL routing; NavLink gets us automatic active state from the URL
and proper back/forward behavior.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The server-side verify_requirements() check inspected the backend's PATH,
but jobs run on the compute node as the user with a different environment.
This produced false negatives (tool present on the cluster but missing on
the backend blocked submission) and false positives (passed on the backend,
failed at runtime).

Add build_requirements_check(), which generates a bash snippet from the same
requirement parser and tool registry. It runs inside the job after PATH/conda/
env setup, checks tool existence and version constraints (sort -V), aggregates
all failures to stderr, and exits non-zero. Failures surface as a FAILED job
with the message in stderr.log, shown by the existing JobDetail UI.

submit_job no longer hard-fails on the server; the check is embedded in the
job script before pre_run/command.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The mutation hooks expose a single shared isPending flag, so passing it to
every ListingCard caused all add/unshare buttons to animate when any one was
in flight. Scope each card's pending state to the listing being acted on by
matching the mutation's in-flight variables (listing_id).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the inline GitHub URL link from catalog cards and add an info button
that opens a ListingInfoDialog, mirroring the "my apps" AppInfoDialog. The
dialog shows the URL, branch, and description in the same table layout, plus
a "Shared by" row, and surfaces Add/Unshare actions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a checkbox to the catalog that filters out listings already in the user's
apps, reusing the existing myAppKeys set. Updates the empty state to explain
when all shared apps are already installed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
The loading state swapped the label for a larger spinner plus loading text,
which grew the button in both dimensions. Now the label stays in normal flow
(hidden in place while loading) and a smaller spinner is overlaid, so the
button keeps a constant size and its label stays vertically centered.

loadingText is now used as the spinner's accessible label. Spinner gains an
optional sizeClasses prop (default unchanged) and skips empty text.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Share/unshare is now only available in the app info dialog. The card keeps
its info, launch, and remove actions; the "Shared" badge still indicates
shared apps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
The trash can icon meant "remove from my apps" on the Apps page but
"unshare" in the catalog. Use users-slash (FaUsersSlash) for unshare and
users (FaUsers) for share, in the catalog card/info dialog and the app info
dialog, to distinguish sharing actions from removal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
- Share/unshare no longer close the app info or catalog info dialogs.
- Fold the share form into AppInfoDialog as an inline view instead of a
  separate stacked dialog, so the dialog stays open throughout sharing
  (stacked Material Tailwind dialogs dismissed the one underneath). Removes
  the now-unused ShareAppDialog.
- Rename the app's "Delete" action to "Remove".
- Add tooltips to every button in both info dialogs and keep the two dialogs
  in sync (layout, button styling, wording).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Apps are versioned on their own terms (package.json, pixi.toml, etc.), so a
manifest-level version is meaningless. Drop it from the AppManifest model and
TS type, the pixi adapter, the info dialog and launch form, and the docs.
Legacy manifests that still include `version:` are accepted but the field is
ignored (Pydantic extra='ignore').

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Requirement checking moved from submit-time on the server to job runtime
in the execution environment, but the Job Execution and extra_paths
sections still described the old server-side behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
submit_job stopped calling verify_requirements when requirement checking
moved to job runtime (build_requirements_check); the function was left
exported and exercised only by tests. Remove it, its sole helper
(_augmented_path), the now-unused shutil/subprocess/packaging imports,
and the orphaned test class. The conda-binary and version-comparison
cases it covered are exercised by the build_requirements_check tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_find_manifests_in_repo raised on the first adapter conversion failure,
so a single adapter's error could mask a later adapter that would have
handled the repo. Collect all adapter errors, return as soon as any
adapter succeeds (logging the rest), and only raise an aggregated error
when no adapter produced a manifest. Add tests covering the
one-fails-one-succeeds, all-fail-aggregated, and none-handle paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
model.py and apps/core.py each defined near-identical requirement-spec
patterns that had to be kept in sync by hand. Make model._REQUIREMENT_PATTERN
the single source of truth (groups: tool, operator, version) and import it
into core. This also lets build_requirements_check read the operator and
version straight from the match, dropping the separate _REQ_OP_PATTERN
split, and tidies merge_requirements to match each spec once.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
published_at comes back from the API as a naive UTC datetime (no tz
marker), but the catalog cards/dialog parsed it with a raw
new Date(...).toLocaleDateString(), which interprets it as local time
and can show the wrong day near midnight. Use the shared formatDateString
helper (already used for job/ticket/link dates), which normalizes naive
strings to UTC before formatting.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The FgButton loading rework stopped rendering loadingText as visible text
and instead exposes it as the spinner overlay's aria-label (role="status"),
keeping the button size constant. The Loading and LoadingWithHref stories
still asserted the text with getByText, so their play functions threw and
Chromatic reported 2 component errors. Query by role/accessible name to
match the current a11y contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TestBuildRequirementsCheck executes the generated requirement-check
snippet through `bash -c` and depends on POSIX tools (grep -oE, sort -V,
arrays) plus chmod-based executable shims. That environment isn't
available/consistent on Windows, so all 12 cases failed on the
windows-latest CI runner. The snippet only ever runs on Linux compute
nodes, so skip the class on win32 (matching test_worker/test_filestore).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant