diff --git a/AGENTS.md b/AGENTS.md index 8686b2f..bcbcdc5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -105,9 +105,10 @@ The release strategy is evidence before expansion: v0.3: evidence pack + assembly gate + provenance checksums v0.4: compare mode for many FASTA files v0.5: submission readiness gate -v0.6: transcriptome profile -v0.7: protein profile -v0.8: reference-panel profile +v0.6: workflow-compatible exit contract +v0.7: transcriptome profile +v0.8: protein profile +v0.9: reference-panel profile later: MCP/tool-agent interface and optional local summaries ``` @@ -123,20 +124,20 @@ Default product boundaries: Recommended next big release: ```text -v0.5 should make submission readiness concrete before adding broad new biological profiles. +v0.6 should simplify workflow integration before adding broad new biological profiles. ``` The next planned feature direction is: ```text -Submission Readiness Gate: --gate submission with --submission-target generic|ncbi. +Workflow-Compatible Exit Contract: successful reports exit 0; report fields carry QC status. ``` -This should stay FASTA-level and database-free. It should check identifier -safety, duplicate first-token IDs, unsafe characters, long identifiers, gap-like -N runs, high ambiguity, and tiny-record advisories. It must not claim repository -acceptance, biological completeness, annotation correctness, or contamination -confirmation. +This should keep JSON and TSV as the source of truth. Successful report +generation should exit `0`, including WARN and FAIL reports. Argument parsing +errors should exit `2`; configuration, input-access, and runtime errors should +exit `3`. Existing v0.5 workflow integrations should keep compatibility handling +until v0.6 packages and containers are published. ## Collaboration Preference diff --git a/Cargo.lock b/Cargo.lock index fda743e..e718225 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -360,7 +360,7 @@ dependencies = [ [[package]] name = "fastaguard" -version = "0.5.0" +version = "0.6.0" dependencies = [ "anyhow", "assert_cmd", diff --git a/Cargo.toml b/Cargo.toml index b738db6..8a54f06 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "fastaguard" -version = "0.5.0" +version = "0.6.0" edition = "2021" license = "MIT" description = "FASTA preflight QC for assembly pipelines" diff --git a/README.md b/README.md index 7b14e13..bf8fd98 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,7 @@ downstream tool. | Channel | Status | | --- | --- | -| Source/package metadata | `v0.5.0` is the latest tagged source release | +| Source/package metadata | This branch prepares `v0.6.0`; `v0.5.0` remains the latest tag | | GitHub release | v0.5 GitHub release binaries are built from the `v0.5.0` tag | | Bioconda | `v0.5.0` is live for Linux and macOS x86_64/ARM64 | | BioContainers | `v0.5.0` is live as a pinned workflow image | @@ -123,6 +123,7 @@ fastaguard --version The `--gate pipeline` examples below require FastaGuard `v0.3.0` or newer. The `fastaguard compare` example requires FastaGuard `v0.4.0` or newer. The `--gate submission` example requires FastaGuard `v0.5.0` or newer. +The conventional exit contract below starts with FastaGuard `v0.6.0`. Run the assembly preflight check: @@ -142,10 +143,10 @@ fastaguard sample.fa --profile assembly --gate pipeline ``` The `pipeline` gate is the v0.3 assembly preset for workflow stop/go decisions. -It fails on duplicate IDs, invalid characters, invalid FASTA structure, and -high-N content. GC and length outliers remain advisory by default because they -are routing signals, not proof of contamination or misassembly. To make an -advisory finding block a pipeline, add it explicitly with `--fail-on`. +It marks duplicate IDs, invalid characters, invalid FASTA structure, and high-N +content as blocking findings. GC and length outliers remain advisory by default +because they are routing signals, not proof of contamination or misassembly. To +mark an advisory finding as blocking, add it explicitly with `--fail-on`. v0.4 compare starter example: @@ -197,15 +198,21 @@ Published BioContainers provides the v0.5 image for workflow engines: docker pull quay.io/biocontainers/fastaguard:0.5.0--hfa8f182_0 ``` -Exit codes: +Starting with FastaGuard v0.6.0, exit codes are: ```text -0 = pass -1 = warnings above configured threshold -2 = hard QC failure -3 = invalid input / tool error +0 = command completed and requested outputs were written +2 = argument parsing error +3 = configuration, input-access, or runtime error ``` +QC PASS/WARN/FAIL decisions are recorded in the machine-readable outputs, +especially `verdict.status`, `gate.status`, and `gate.blocking_findings`. +Workflow engines should route on those fields instead of interpreting QC +findings from the process exit code. Single-file TSV reports include +`input_path`, `verdict`, and `gate_status`; compare TSV reports retain one row +per input with its path and status fields. + ## Product Thesis FASTA files are everywhere, but FASTA QC is fragmented across ad hoc scripts, `seqkit stats`, assembly QC tools, completeness tools, contamination workflows, and pipeline-specific checks. Each is useful, but none is the simple default first command for: @@ -261,7 +268,7 @@ v0.2 expands the assembly preflight layer with: v0.3 adds the assembly gate contract: -- `--gate pipeline` for default workflow blocking behavior +- `--gate pipeline` for the default workflow gate policy - `gate.blocking_findings` for machine stop/go decisions - checksum provenance with `provenance.input_sha256` - explicit advisory findings for evidence that should route follow-up QC rather @@ -285,6 +292,15 @@ v0.5 adds the submission-readiness gate: - boundaries that keep FastaGuard upstream of official validators, NCBI FCS, annotation validation, QUAST, BUSCO, BlobToolKit, and CheckM +v0.6 makes report generation workflow-compatible: + +- successful report generation exits `0` for PASS, WARN, and FAIL reports +- argument parsing errors exit `2`; configuration, input-access, and runtime + errors exit `3` +- single-file TSV reports include `input_path` for downstream routing +- workflows enforce QC policy from stable report fields instead of process + status + ## Positioning FastaGuard should recommend deeper tools when they are appropriate: @@ -324,6 +340,7 @@ FastaGuard catches FASTA-level assembly problems before expensive assembly QC. - [v0.5 submission readiness evidence](docs/evidence/fastaguard-v0.5-submission-readiness.md) - [v0.5 public evidence workflow](docs/evidence/fastaguard-v0.5-public-evidence.md) - [Packaging](docs/packaging.md) +- [v0.6.0 release notes](docs/releases/v0.6.0.md) - [v0.5.0 release notes](docs/releases/v0.5.0.md) - [v0.4.0 release notes](docs/releases/v0.4.0.md) - [v0.3.0 release notes](docs/releases/v0.3.0.md) diff --git a/docs/mvp-spec.md b/docs/mvp-spec.md index aaf285d..9c8ddc9 100644 --- a/docs/mvp-spec.md +++ b/docs/mvp-spec.md @@ -150,13 +150,19 @@ Default WARN conditions: ## Exit Codes +Starting with FastaGuard v0.6.0: + ```text -0 = pass -1 = warnings above configured threshold -2 = hard QC failure -3 = invalid input / tool error +0 = command completed and requested outputs were written +2 = argument parsing error +3 = configuration, input-access, or runtime error ``` +QC PASS/WARN/FAIL decisions are report fields, not process-failure signals. +Pipelines should read `verdict.status`, `gate.status`, and +`gate.blocking_findings` from JSON/TSV outputs. Single-file TSV reports include +`input_path`, `verdict`, and `gate_status` for downstream routing. + ## Success Criteria The first release is successful if: @@ -164,7 +170,7 @@ The first release is successful if: - it validates huge FASTA files without loading full sequences into memory - it produces useful HTML, JSON, TSV, and MultiQC-compatible outputs - it catches invalid FASTA structure, duplicate IDs, invalid characters, and high-N content -- it has deterministic, documented exit codes +- it has deterministic, documented status fields and tool-error exit codes - it can be added to a Nextflow or Snakemake pipeline in under 5 minutes ## Implementation Status @@ -174,5 +180,5 @@ The v0.1 assembly MVP is implemented as a Rust CLI with: - streaming FASTA parsing for plain and gzipped files - assembly metrics - explainable findings -- deterministic verdict exit codes +- deterministic verdict status fields - JSON, TSV, HTML, and MultiQC-compatible outputs diff --git a/docs/output-contract.md b/docs/output-contract.md index 72f9efc..5a2d11b 100644 --- a/docs/output-contract.md +++ b/docs/output-contract.md @@ -4,7 +4,7 @@ FastaGuard should be pipeline-native from the first release. -The output contract is as important as the HTML report. Pipeline authors need stable field names, deterministic behavior, documented exit codes, and versioned schemas. +The output contract is as important as the HTML report. Pipeline authors need stable field names, deterministic behavior, documented tool-error exit codes, and versioned schemas. ## Artifacts @@ -29,6 +29,23 @@ fastaguard_compare_mqc.json v0.5 adds submission-readiness gate fields to the same JSON, TSV, HTML, MultiQC, and compare artifacts. JSON remains the source of truth. +v0.6 separates process execution from QC status. Successful report generation +returns exit code `0` for PASS, WARN, and FAIL reports. + +## Process Exit Contract + +Starting with FastaGuard v0.6.0: + +```text +0 = command completed and requested outputs were written +2 = argument parsing error +3 = configuration, input-access, or runtime error +``` + +Workflow engines should apply stop/go policy from `gate.status`, +`gate.blocking_findings`, or `verdict.status`. Process status only indicates +whether FastaGuard completed the requested command and reports. + ## JSON Contract Example v0.3 shape: @@ -416,7 +433,8 @@ Recommended first rows: ```text metric value -schema_version 0.3.0 +schema_version 0.5.0 +input_path sample.fa profile assembly verdict FAIL gate_mode pipeline diff --git a/docs/packaging.md b/docs/packaging.md index aafaea4..7d0a0c8 100644 --- a/docs/packaging.md +++ b/docs/packaging.md @@ -135,7 +135,7 @@ Keep future releases compatible with Bioconda expectations: - keep a single static-ish CLI binary target - keep deterministic tests and tiny fixtures - avoid runtime databases for early releases -- maintain stable exit codes +- keep process exit codes reserved for CLI/tool execution errors - maintain a versioned JSON Schema Bioconda recipe guidance checked for this setup: diff --git a/docs/releases/v0.6.0.md b/docs/releases/v0.6.0.md new file mode 100644 index 0000000..41c1066 --- /dev/null +++ b/docs/releases/v0.6.0.md @@ -0,0 +1,31 @@ +# FastaGuard v0.6.0 + +## Workflow-Compatible Exit Contract + +FastaGuard now separates command execution from FASTA quality decisions. +Successful report generation exits with code `0`, including reports whose +verdict or gate status is `WARN` or `FAIL`. Argument parsing errors exit with +code `2`, while configuration, input-access, and runtime errors exit with code +`3`. + +QC decisions remain explicit in the report contract: + +- `verdict.status` records the overall PASS/WARN/FAIL result. +- `gate.status` records the selected gate result. +- `gate.blocking_findings` lists findings that should stop downstream work. +- single-file TSV reports include `input_path`, `verdict`, and `gate_status`. +- compare TSV reports retain one row per input with its path and status fields. + +This keeps JSON and TSV reports available to Nextflow, Snakemake, nf-core, and +other workflow engines. Pipelines can publish the evidence first and apply +their own stop/go policy downstream. + +## Migration + +Workflows that previously interpreted process exit codes `1` and `2` as WARN +and FAIL should instead read `gate.status` or `verdict.status` from JSON or TSV. +The repository includes `examples/workflows/check_fastaguard_gate.py` for +workflows that need a separate enforcement step after reports are collected. + +The JSON schema and finding catalog remain at version `0.5.0` because their +structures and finding semantics are unchanged in this release. diff --git a/docs/roadmap.md b/docs/roadmap.md index 361a3d9..4209fee 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -27,7 +27,7 @@ Capabilities: - GC-vs-length plot data and HTML plot - PASS / WARN / FAIL verdict - JSON, TSV, HTML, and MultiQC-compatible outputs -- deterministic exit codes +- deterministic report status fields ## v0.2: Assembly Trust + Pipeline Adoption @@ -118,7 +118,25 @@ Development scope: - FastaGuard does not replace NCBI, ENA, DDBJ official validators or guarantee repository acceptance -## v0.6: Transcriptome Profile +## v0.6: Workflow-Compatible Exit Contract + +Goal: + +```text +Separate command execution from FASTA quality decisions in workflow engines. +``` + +Development scope: + +- return exit code `0` whenever requested reports are written successfully, + including WARN and FAIL reports +- reserve exit code `2` for argument parsing errors +- reserve exit code `3` for configuration, input-access, and runtime errors +- keep PASS/WARN/FAIL decisions in stable JSON and TSV report fields +- include `input_path` in single-file TSV reports for downstream routing +- document migration from legacy QC-derived process exit codes + +## v0.7: Transcriptome Profile Potential additions: @@ -128,7 +146,7 @@ Potential additions: - extreme GC outliers - isoform-heavy warning heuristics -## v0.7: Protein Profile +## v0.8: Protein Profile Potential additions: @@ -138,7 +156,7 @@ Potential additions: - low-complexity regions - suspicious nucleotide-looking proteins -## v0.8: Reference Panel Profile +## v0.9: Reference Panel Profile Potential additions: diff --git a/docs/tool-landscape.md b/docs/tool-landscape.md index 255d828..d98305f 100644 --- a/docs/tool-landscape.md +++ b/docs/tool-landscape.md @@ -56,7 +56,7 @@ Without FastaGuard, users typically combine several partial checks: - run QUAST for assembly metrics - run BUSCO for biological completeness - run BlobToolKit or taxonomy tooling for contamination exploration -- rely on pipeline-specific assumptions for exit codes and report parsing +- rely on pipeline-specific assumptions for report parsing and stop/go policy That works, but it is fragmented. The missing layer is a default, explainable, machine-readable FASTA preflight contract. diff --git a/docs/use-cases.md b/docs/use-cases.md index c8c3722..f77b4e2 100644 --- a/docs/use-cases.md +++ b/docs/use-cases.md @@ -56,11 +56,12 @@ spend time on bad input. ### Pipeline Gate For Workflow Engines FastaGuard gives Nextflow, Snakemake, nf-core, Galaxy, and institutional -pipelines a simple PASS/WARN/FAIL decision with deterministic exit codes and -structured evidence. +pipelines a simple PASS/WARN/FAIL decision with structured evidence in JSON, +TSV, HTML, and MultiQC-compatible outputs. Pipelines can stop early on hard FASTA failures, continue with warnings, or -route samples to follow-up tools based on machine-readable findings. +route samples to follow-up tools based on machine-readable fields such as +`gate.status` and `gate.blocking_findings`. ### Batch Triage With Compare Mode diff --git a/docs/vision-plan.md b/docs/vision-plan.md index 0ae8b51..bed45ec 100644 --- a/docs/vision-plan.md +++ b/docs/vision-plan.md @@ -26,7 +26,7 @@ The release strategy is evidence before expansion. FastaGuard should not rush into many biological profiles until the assembly preflight contract is trusted. The product should earn adoption in this order: -1. **Trust:** reproducible evidence, stable schemas, clear exit codes, installable packages. +1. **Trust:** reproducible evidence, stable schemas, clear status fields, installable packages. 2. **Integration:** Bioconda, BioContainers, MultiQC, Nextflow, Snakemake, Galaxy. 3. **Scale:** compare mode for many FASTA files and batch pipeline reports. 4. **Readiness depth:** submission-oriented preflight checks before official validators. @@ -109,7 +109,25 @@ The v0.5 contract should make workflow routing explicit through `gate.mode`, `readiness.categories[id=submission]` record, while still pointing users to official validators for final repository-specific checks. -### v0.6: Transcriptome Profile +### v0.6: Workflow-Compatible Exit Contract + +Goal: + +```text +Let workflows collect complete QC reports before applying their own stop/go policy. +``` + +Successful report generation should exit with code `0` for PASS, WARN, and FAIL +reports. Argument parsing errors should use code `2`; configuration, +input-access, and runtime errors should use code `3`. JSON and TSV remain the +source of truth for `verdict.status`, `gate.status`, and blocking findings. + +Single-file TSV reports should include `input_path` alongside status fields so +workflow engines can route samples without scraping logs or HTML. Existing +workflow examples pinned to v0.5 should retain compatibility handling until a +v0.6 package and container are published. + +### v0.7: Transcriptome Profile Goal: @@ -129,7 +147,7 @@ FastaGuard should not claim transcriptome biological completeness. It should route users to transcriptome-specific completeness and annotation tools when needed. -### v0.7: Protein Profile +### v0.8: Protein Profile Goal: @@ -148,7 +166,7 @@ Initial protein checks: Protein mode should be strict about alphabet validity and careful about biology: it should flag preflight problems, not infer functional correctness. -### v0.8: Reference-Panel Profile +### v0.9: Reference-Panel Profile Goal: @@ -219,7 +237,7 @@ Required adoption qualities: - generated BioContainers image - stable JSON schema - deterministic outputs -- clear exit codes +- clear tool-error exit codes - MultiQC compatibility - Nextflow, nf-core, Snakemake, and Galaxy examples - small public evidence pack @@ -239,12 +257,13 @@ Recommended sequence: v0.3: evidence pack + assembly gate + provenance checksums v0.4: compare mode for many FASTA files v0.5: submission readiness gate -v0.6: transcriptome profile -v0.7: protein profile -v0.8: reference-panel profile +v0.6: workflow-compatible exit contract +v0.7: transcriptome profile +v0.8: protein profile +v0.9: reference-panel profile later: MCP/tool-agent interface and optional local summaries ``` This path gives FastaGuard the best chance to become a default tool: prove the -assembly gate first, scale to batches, make submission readiness concrete, then -expand profiles. +assembly gate first, scale to batches, make submission readiness concrete, +simplify workflow integration, then expand profiles. diff --git a/docs/workflow-readiness.md b/docs/workflow-readiness.md index b466007..fbe9976 100644 --- a/docs/workflow-readiness.md +++ b/docs/workflow-readiness.md @@ -32,6 +32,12 @@ External upstream-style validation was run on 2026-07-03 in dedicated checkouts: PASS, WARN, FAIL, and invalid FASTA fixtures plus captured `exit_code` outputs. +The v0.6 source contract removes QC-derived process failures: successfully +written PASS, WARN, and FAIL reports all return exit code `0`. The pinned v0.5 +starters keep their compatibility capture until v0.6 packages and containers +are published. After publication, upstream wrappers can remove that workaround +and route directly from JSON or TSV status fields. + ## Safe Order 1. Run local repository tests first. @@ -53,11 +59,11 @@ official submission validators. The default workflow pattern is: collect FASTA-level evidence -> apply stop/go policy -> route downstream tools ``` -For local fail-fast runs, `--gate pipeline` and `--gate submission` write JSON, -TSV, HTML, and MultiQC-compatible evidence before returning a blocking exit -code. Some workflow engines remove failed-job outputs by default, so production -integrations should use a collect-then-gate pattern when evidence preservation -matters. The important contract fields are: +With the published v0.5 runtime, `--gate pipeline` and `--gate submission` write +JSON, TSV, HTML, and MultiQC-compatible evidence before returning a QC-derived +exit code. The pinned local starters capture that code so the evidence remains +available, then leave stop/go enforcement to a downstream gate step. The +important contract fields are: - `verdict.status` - `gate.mode` diff --git a/examples/nextflow/main.nf b/examples/nextflow/main.nf index efad50d..f98f526 100644 --- a/examples/nextflow/main.nf +++ b/examples/nextflow/main.nf @@ -14,8 +14,8 @@ process FASTAGUARD { script: """ - # Fail-fast starter example: FastaGuard WARN exits 1 and FAIL exits 2. - # Depending on engine behavior, evidence may remain only in the work directory. + # FastaGuard writes PASS/WARN/FAIL decisions to JSON/TSV/HTML reports. + # Route downstream workflow steps by reading gate.status from JSON. fastaguard ${fasta} \ --profile assembly \ --gate pipeline \ @@ -32,6 +32,6 @@ workflow { // Compare mode starter pattern for v0.4 cohort triage: // fastaguard compare assemblies/*.fa --profile assembly --gate pipeline -// This is local fail-fast starter guidance, not an upstream nf-core submission yet. -// FastaGuard WARN exits 1 and FAIL exits 2; collect-then-gate wrappers can preserve -// evidence in publish directories before applying workflow stop/go logic. +// This is local starter guidance, not an upstream nf-core submission yet. +// FastaGuard returns success when reports are written; collect-then-gate wrappers +// can apply workflow stop/go logic from gate.status. diff --git a/examples/nf-core/README.md b/examples/nf-core/README.md index ac58df2..329a50b 100644 --- a/examples/nf-core/README.md +++ b/examples/nf-core/README.md @@ -26,6 +26,7 @@ Emitted outputs: - `json` - `tsv` - `mqc` +- `exit_code` - `versions` The module assumes `fastaguard` is available on `PATH` when run without a @@ -47,13 +48,12 @@ The command block is written for the v0.3 assembly gate and runs: fastaguard sample.fa --profile assembly --gate pipeline ``` -That gate contract blocks downstream workflow steps on duplicate IDs, invalid -characters, invalid FASTA structure, and high-N content. Gate failures intentionally exit with code `2` after writing reports, so downstream workflow steps stop while the JSON/HTML evidence remains available. - -This is a local fail-fast starter example: FastaGuard WARN exits 1 and FAIL exits 2. -Depending on workflow engine behavior, naive fail-fast runs may leave evidence -only in the work directory. A production wrapper can use a collect-then-gate -pattern to publish JSON/HTML/TSV/MultiQC evidence before applying stop/go logic. +That gate contract marks duplicate IDs, invalid characters, invalid FASTA +structure, and high-N content as blocking findings. The published v0.5.0 +runtime returns `1` for WARN and `2` for FAIL. This module captures that legacy +status so JSON, HTML, TSV, and MultiQC evidence can be published; a downstream +gate step should then enforce stop/go policy from `gate.status` or the captured +exit code. Runtime errors with status `3` still fail the process. For v0.4 cohort triage, compare mode is a starter pattern: @@ -75,9 +75,9 @@ Pipeline authors should route on: - `gate.blocking_findings` - `readiness.categories[id=submission]` -This is local fail-fast starter guidance only. It is not yet an upstream nf-core -module submission, and the Snakemake example is not yet an official wrapper -submission. +This is local collect-then-gate starter guidance only. It is not yet an +upstream nf-core module submission, and the Snakemake example is not yet an +official wrapper submission. Example include: diff --git a/examples/reports/README.md b/examples/reports/README.md index 2344658..8b28554 100644 --- a/examples/reports/README.md +++ b/examples/reports/README.md @@ -1,6 +1,6 @@ # Example Reports -These tiny examples show the current v0.5 output contract without requiring large datasets. +These tiny examples show the current v0.6 output contract without requiring large datasets. ## Assembly Pass @@ -41,4 +41,6 @@ cargo run -- testdata/problem_assembly.fa \ --multiqc examples/reports/assembly_fail/fastaguard_mqc.json ``` -The command exits with code `2` because this example contains critical FASTA-level blockers. Inspect the JSON `gate` object and readiness categories to separate blocking findings from advisory findings. +The command exits with code `0` after writing the FAIL report. Inspect the JSON +`gate` object and readiness categories to separate blocking findings from +advisory findings. diff --git a/examples/reports/assembly_fail/fastaguard.json b/examples/reports/assembly_fail/fastaguard.json index bc738c8..ba08540 100644 --- a/examples/reports/assembly_fail/fastaguard.json +++ b/examples/reports/assembly_fail/fastaguard.json @@ -2,7 +2,7 @@ "schema_version": "0.5.0", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "path": "testdata/problem_assembly.fa", diff --git a/examples/reports/assembly_fail/fastaguard.tsv b/examples/reports/assembly_fail/fastaguard.tsv index 59d824c..6f9540a 100644 --- a/examples/reports/assembly_fail/fastaguard.tsv +++ b/examples/reports/assembly_fail/fastaguard.tsv @@ -1,5 +1,6 @@ metric value schema_version 0.5.0 +input_path testdata/problem_assembly.fa profile assembly verdict FAIL gate_mode none diff --git a/examples/reports/assembly_fail/fastaguard_report.html b/examples/reports/assembly_fail/fastaguard_report.html index 5e08b79..1e66cf3 100644 --- a/examples/reports/assembly_fail/fastaguard_report.html +++ b/examples/reports/assembly_fail/fastaguard_report.html @@ -579,7 +579,7 @@

JSON

"schema_version": "0.5.0", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "path": "testdata/problem_assembly.fa", diff --git a/examples/reports/assembly_pass/fastaguard.json b/examples/reports/assembly_pass/fastaguard.json index 57ae492..37d4285 100644 --- a/examples/reports/assembly_pass/fastaguard.json +++ b/examples/reports/assembly_pass/fastaguard.json @@ -2,7 +2,7 @@ "schema_version": "0.5.0", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "path": "testdata/valid_assembly.fa", diff --git a/examples/reports/assembly_pass/fastaguard.tsv b/examples/reports/assembly_pass/fastaguard.tsv index b858d05..3e1c88a 100644 --- a/examples/reports/assembly_pass/fastaguard.tsv +++ b/examples/reports/assembly_pass/fastaguard.tsv @@ -1,5 +1,6 @@ metric value schema_version 0.5.0 +input_path testdata/valid_assembly.fa profile assembly verdict WARN gate_mode none diff --git a/examples/reports/assembly_pass/fastaguard_report.html b/examples/reports/assembly_pass/fastaguard_report.html index fbc1b28..312d0fd 100644 --- a/examples/reports/assembly_pass/fastaguard_report.html +++ b/examples/reports/assembly_pass/fastaguard_report.html @@ -201,7 +201,7 @@

JSON

"schema_version": "0.5.0", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "path": "testdata/valid_assembly.fa", diff --git a/examples/snakemake/Snakefile b/examples/snakemake/Snakefile index 0c85bb2..b0f0bb0 100644 --- a/examples/snakemake/Snakefile +++ b/examples/snakemake/Snakefile @@ -8,8 +8,8 @@ rule fastaguard: multiqc="fastaguard_mqc.json" shell: """ - # Fail-fast starter example: FastaGuard WARN exits 1 and FAIL exits 2. - # Depending on engine behavior, evidence may remain only in the work directory. + # FastaGuard writes PASS/WARN/FAIL decisions to JSON/TSV/HTML reports. + # Route downstream workflow steps by reading gate.status from JSON. fastaguard {input.fasta} \ --profile assembly \ --gate pipeline \ @@ -21,6 +21,6 @@ rule fastaguard: # Compare mode starter pattern for v0.4 cohort triage: # fastaguard compare assemblies/*.fa --profile assembly --gate pipeline -# This is local fail-fast starter guidance, not an official Snakemake submission yet. -# FastaGuard WARN exits 1 and FAIL exits 2; collect-then-gate wrappers can preserve -# evidence in publish directories before applying workflow stop/go logic. +# This is local starter guidance, not an official Snakemake submission yet. +# FastaGuard returns success when reports are written; collect-then-gate wrappers +# can apply workflow stop/go logic from gate.status. diff --git a/src/cli.rs b/src/cli.rs index 8eaca73..6fd2467 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -83,7 +83,7 @@ pub struct AnalysisArgs { #[arg(long, default_value = "assembly")] pub profile: String, - /// Gate preset for pipeline-friendly failure behavior. + /// Gate preset for report blocking policy. #[arg(long, value_enum, default_value_t = GateMode::None)] pub gate: GateMode, @@ -91,7 +91,7 @@ pub struct AnalysisArgs { #[arg(long, value_enum)] pub submission_target: Option, - /// Comma-separated rule IDs that should fail the run when triggered. + /// Comma-separated rule IDs that mark the report as FAIL when triggered. #[arg(long, value_delimiter = ',')] pub fail_on: Vec, diff --git a/src/compare.rs b/src/compare.rs index 7de89f0..e38c989 100644 --- a/src/compare.rs +++ b/src/compare.rs @@ -20,7 +20,6 @@ pub fn run_compare(config: CompareConfig) -> Result { } let summary = compare_summary(&samples); - let worst = worst_status(samples.iter().map(|sample| sample.gate_status)); let cohort_findings = cohort_findings(&samples); let report = CompareReport { schema_version: SCHEMA_VERSION.to_string(), @@ -39,7 +38,7 @@ pub fn run_compare(config: CompareConfig) -> Result { }; crate::report::write_compare_all(&report, &config.outputs)?; - Ok(exit_code(worst)) + Ok(0) } fn validate_unique_sample_ids(inputs: &[std::path::PathBuf]) -> Result<()> { @@ -244,27 +243,6 @@ pub(crate) fn sample_id(path: &Path) -> String { .to_string() } -pub(crate) fn worst_status(statuses: I) -> VerdictStatus -where - I: IntoIterator, -{ - statuses - .into_iter() - .fold(VerdictStatus::Pass, |worst, status| match (worst, status) { - (VerdictStatus::Fail, _) | (_, VerdictStatus::Fail) => VerdictStatus::Fail, - (VerdictStatus::Warn, _) | (_, VerdictStatus::Warn) => VerdictStatus::Warn, - _ => VerdictStatus::Pass, - }) -} - -fn exit_code(status: VerdictStatus) -> i32 { - match status { - VerdictStatus::Pass => 0, - VerdictStatus::Warn => 1, - VerdictStatus::Fail => 2, - } -} - fn usize_to_u64(value: usize) -> u64 { value.try_into().unwrap_or(u64::MAX) } @@ -275,23 +253,6 @@ mod tests { use super::*; - #[test] - fn worst_status_prefers_fail_over_warn_over_pass() { - assert_eq!( - worst_status([ - VerdictStatus::Pass, - VerdictStatus::Fail, - VerdictStatus::Warn - ]), - VerdictStatus::Fail - ); - assert_eq!( - worst_status([VerdictStatus::Pass, VerdictStatus::Warn]), - VerdictStatus::Warn - ); - assert_eq!(worst_status([VerdictStatus::Pass]), VerdictStatus::Pass); - } - #[test] fn sample_id_uses_file_stem_without_compression_suffix() { assert_eq!(sample_id(Path::new("assemblies/ecoli.fa")), "ecoli"); diff --git a/src/lib.rs b/src/lib.rs index 9b19c79..bbb596b 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -43,7 +43,7 @@ fn run_single(config: cli::RunConfig) -> Result { let outputs = config.outputs.clone(); let output = build_single_report(config, run_started)?; report::write_all(&output, &outputs)?; - Ok(output.exit_code()) + Ok(0) } pub(crate) fn build_single_report( diff --git a/src/models.rs b/src/models.rs index 3af5443..c30c5b9 100644 --- a/src/models.rs +++ b/src/models.rs @@ -521,14 +521,6 @@ impl FastaguardReport { }, }) } - - pub fn exit_code(&self) -> i32 { - match self.verdict.status { - VerdictStatus::Pass => 0, - VerdictStatus::Warn => 1, - VerdictStatus::Fail => 2, - } - } } pub fn finding_actions(id: &str) -> Vec { diff --git a/src/report/tsv.rs b/src/report/tsv.rs index da5d817..4c73ab4 100644 --- a/src/report/tsv.rs +++ b/src/report/tsv.rs @@ -12,6 +12,11 @@ pub fn write(report: &FastaguardReport, path: &Path) -> Result<()> { writeln!(writer, "metric\tvalue")?; write_metric(&mut writer, "schema_version", &report.schema_version)?; + write_metric( + &mut writer, + "input_path", + sanitize_tsv_value(&report.input.path), + )?; write_metric(&mut writer, "profile", &report.input.profile)?; write_metric( &mut writer, @@ -210,6 +215,16 @@ fn write_metric( } } +fn sanitize_tsv_value(value: &str) -> String { + value + .chars() + .map(|character| match character { + '\t' | '\r' | '\n' => ' ', + _ => character, + }) + .collect() +} + fn affected_record_count(report: &FastaguardReport, finding_id: &str) -> u64 { report .findings @@ -292,6 +307,40 @@ mod tests { assert!(output.contains("verdict\tWARN\n"), "{output}"); } + #[test] + fn writes_input_path_for_downstream_routing() { + let report = test_report(VerdictStatus::Warn); + let file = NamedTempFile::new().unwrap(); + + write(&report, file.path()).unwrap(); + + let output = fs::read_to_string(file.path()).unwrap(); + assert!(output.contains("input_path\tinput.fa\n"), "{output}"); + } + + #[test] + fn sanitizes_input_path_without_adding_tsv_rows() { + let mut report = test_report(VerdictStatus::Warn); + report.input.path = "inputs/sample\tone\n.fa".to_string(); + let file = NamedTempFile::new().unwrap(); + + write(&report, file.path()).unwrap(); + + let output = fs::read_to_string(file.path()).unwrap(); + assert!( + output.contains("input_path\tinputs/sample one .fa\n"), + "{output}" + ); + assert_eq!( + output + .lines() + .filter(|line| line.starts_with("input_path\t")) + .count(), + 1, + "{output}" + ); + } + #[test] fn writes_outlier_counts_from_matching_findings() { let mut report = test_report(VerdictStatus::Warn); diff --git a/tests/cli.rs b/tests/cli.rs index e917bef..7e9efca 100644 --- a/tests/cli.rs +++ b/tests/cli.rs @@ -17,6 +17,20 @@ fn help_mentions_preflight_positioning() { .stdout(predicate::str::contains("FASTA preflight QC")); } +#[test] +fn help_describes_gate_options_as_report_policy() { + let mut cmd = Command::cargo_bin("fastaguard").unwrap(); + cmd.arg("--help") + .assert() + .success() + .stdout(predicate::str::contains( + "Gate preset for report blocking policy", + )) + .stdout(predicate::str::contains("mark the report as FAIL")) + .stdout(predicate::str::contains("failure behavior").not()) + .stdout(predicate::str::contains("fail the run").not()); +} + #[test] fn help_does_not_advertise_removed_warning_flag() { let removed_flag = ["--warn", "-on"].concat(); @@ -155,7 +169,7 @@ fn compare_writes_json_with_mixed_status_samples() { .arg("--multiqc") .arg(&multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -219,7 +233,7 @@ fn compare_golden_mixed_status_matches() { .arg("--multiqc") .arg(&paths.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); assert_json_matches_golden(&paths.json, "tests/golden/compare_mixed_status.json"); @@ -324,7 +338,7 @@ fn compare_includes_structurally_invalid_fasta_sample() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -361,7 +375,7 @@ fn valid_assembly_writes_all_outputs_and_warns_for_terminal_ns() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(1) + .success() .stderr(predicate::str::is_empty()); assert_all_outputs_exist(&outputs); @@ -390,7 +404,7 @@ fn valid_report_includes_machine_summary_scope_and_provenance() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(1) + .success() .stderr(predicate::str::is_empty()); let report = read_json(&outputs.json); @@ -448,7 +462,7 @@ fn report_includes_v0_4_provenance_and_routing_hints() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(1); + .success(); let report = read_json(&outputs.json); assert_eq!(report["schema_version"], json!("0.5.0")); @@ -500,7 +514,7 @@ fn valid_report_includes_plot_contract() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(1) + .success() .stderr(predicate::str::is_empty()); let report = read_json(&outputs.json); @@ -557,7 +571,7 @@ fn gc_outlier_plot_flags_are_backed_by_warning_finding() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(1) + .success() .stderr(predicate::str::is_empty()); let report = read_json(&outputs.json); @@ -610,7 +624,7 @@ fn assembly_outliers_are_promoted_to_findings_without_fail_by_default() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(1) + .success() .stderr(predicate::str::is_empty()); let report = read_json(&outputs.json); @@ -705,14 +719,14 @@ fn valid_assembly_json_matches_golden_contract() { .arg("--multiqc") .arg(&paths.multiqc) .assert() - .code(1) + .success() .stderr(predicate::str::is_empty()); assert_json_matches_golden(&paths.json, "tests/golden/valid_assembly.json"); } #[test] -fn problem_assembly_returns_failure_for_default_critical_findings() { +fn problem_assembly_writes_failure_report_with_successful_process() { let temp_dir = TempDir::new().unwrap(); let outputs = output_paths(&temp_dir, "problem"); @@ -726,7 +740,7 @@ fn problem_assembly_returns_failure_for_default_critical_findings() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); assert_all_outputs_exist(&outputs); @@ -752,7 +766,7 @@ fn pipeline_gate_report_lists_blocking_and_advisory_findings() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -809,7 +823,7 @@ fn report_includes_readiness_matrix() { multiqc.to_str().unwrap(), ]) .assert() - .code(2); + .success(); let report: serde_json::Value = serde_json::from_str(&std::fs::read_to_string(json).unwrap()).unwrap(); @@ -845,7 +859,7 @@ fn html_report_shows_gate_decision() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let html = std::fs::read_to_string(&outputs.html).unwrap(); @@ -876,7 +890,7 @@ fn gate_none_report_preserves_warning_behavior_and_checksum() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -920,7 +934,7 @@ fn problem_assembly_json_matches_golden_contract() { .arg("--multiqc") .arg(&paths.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); assert_json_matches_golden(&paths.json, "tests/golden/problem_assembly.json"); @@ -941,7 +955,7 @@ fn problem_report_includes_structured_finding_actions() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -985,7 +999,7 @@ fn problem_report_includes_v0_2_finding_taxonomy() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -1011,7 +1025,7 @@ fn problem_report_includes_per_record_evidence() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -1068,14 +1082,14 @@ fn invalid_fasta_json_matches_golden_contract() { .arg("--multiqc") .arg(&paths.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); assert_json_matches_golden(&paths.json, "tests/golden/invalid_empty_record.json"); } #[test] -fn structurally_invalid_fasta_returns_failure_report() { +fn structurally_invalid_fasta_writes_failure_report_with_successful_process() { let temp_dir = TempDir::new().unwrap(); let input = temp_dir.path().join("invalid.fa"); std::fs::write(&input, ">empty\n").unwrap(); @@ -1092,7 +1106,7 @@ fn structurally_invalid_fasta_returns_failure_report() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); assert_all_outputs_exist(&outputs); @@ -1119,7 +1133,7 @@ fn invalid_fasta_report_includes_machine_contract_fields() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -1171,7 +1185,7 @@ fn unknown_gate_value_is_cli_error() { let mut cmd = Command::cargo_bin("fastaguard").unwrap(); cmd.args(["testdata/valid_assembly.fa", "--gate", "strict"]) .assert() - .failure() + .code(2) .stderr(predicate::str::contains("invalid value 'strict'")); } @@ -1209,7 +1223,7 @@ fn submission_gate_defaults_to_generic_target() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(1) + .success() .stderr(predicate::str::is_empty()); let report = read_json(&outputs.json); @@ -1242,7 +1256,7 @@ fn submission_target_ncbi_is_serialized_when_requested() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(1) + .success() .stderr(predicate::str::is_empty()); let report = read_json(&outputs.json); @@ -1280,7 +1294,7 @@ fn submission_gate_fails_identifier_hazards() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2) + .success() .stderr(predicate::str::contains("fastaguard error:").not()); let report = read_json(&outputs.json); @@ -1336,7 +1350,7 @@ fn submission_gate_invalid_chars_fail_submission_readiness() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2); + .success(); let report = read_json(&outputs.json); let submission_readiness = report["readiness"]["categories"] @@ -1378,7 +1392,7 @@ fn submission_identifier_hazards_route_to_official_validators_without_claiming_r .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2); + .success(); let report = read_json(&outputs.json); assert_routing_hint( @@ -1415,7 +1429,7 @@ fn submission_gate_outputs_tsv_multiqc_and_html_fields() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2); + .success(); let tsv = std::fs::read_to_string(&outputs.tsv).unwrap(); assert!(tsv.contains("submission_target\tncbi\n"), "{tsv}"); @@ -1466,7 +1480,7 @@ fn compare_submission_gate_aggregates_submission_status() { .arg("--multiqc") .arg(&outputs.multiqc) .assert() - .code(2); + .success(); let report = read_json(&outputs.json); assert_eq!(report["summary"]["submission_fail_count"], json!(1)); diff --git a/tests/golden/compare_all_pass.json b/tests/golden/compare_all_pass.json index e149e03..13a5012 100644 --- a/tests/golden/compare_all_pass.json +++ b/tests/golden/compare_all_pass.json @@ -3,7 +3,7 @@ "report_type": "compare", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "profile": "assembly", diff --git a/tests/golden/compare_mixed_status.json b/tests/golden/compare_mixed_status.json index 9f75bca..42ff95f 100644 --- a/tests/golden/compare_mixed_status.json +++ b/tests/golden/compare_mixed_status.json @@ -3,7 +3,7 @@ "report_type": "compare", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "profile": "assembly", diff --git a/tests/golden/invalid_empty_record.json b/tests/golden/invalid_empty_record.json index 42c81e1..405559d 100644 --- a/tests/golden/invalid_empty_record.json +++ b/tests/golden/invalid_empty_record.json @@ -2,7 +2,7 @@ "schema_version": "0.5.0", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "path": "testdata/invalid_empty_record.fa", diff --git a/tests/golden/problem_assembly.json b/tests/golden/problem_assembly.json index 0056662..49a7f2b 100644 --- a/tests/golden/problem_assembly.json +++ b/tests/golden/problem_assembly.json @@ -2,7 +2,7 @@ "schema_version": "0.5.0", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "path": "testdata/problem_assembly.fa", diff --git a/tests/golden/valid_assembly.json b/tests/golden/valid_assembly.json index ba793f1..998471f 100644 --- a/tests/golden/valid_assembly.json +++ b/tests/golden/valid_assembly.json @@ -2,7 +2,7 @@ "schema_version": "0.5.0", "tool": { "name": "FastaGuard", - "version": "0.5.0" + "version": "0.6.0" }, "input": { "path": "testdata/valid_assembly.fa", diff --git a/tests/python/test_adoption_assets.py b/tests/python/test_adoption_assets.py index 2e872b8..3e65f9f 100644 --- a/tests/python/test_adoption_assets.py +++ b/tests/python/test_adoption_assets.py @@ -84,7 +84,8 @@ def test_v0_3_gate_examples_do_not_pin_v0_2_runtimes(self): ) self.assertNotIn("fastaguard=0.2.0", wrapper_env) self.assertIn("--gate {gate}", wrapper_py) - self.assertIn("Gate failures intentionally exit with code `2`", nf_core_readme) + self.assertIn("published v0.5.0", nf_core_readme) + self.assertIn("captures that legacy", nf_core_readme) self.assertIn("fastaguard.exit_code", snakemake_readme) def test_v0_4_docs_explain_preflight_readiness_and_compare_mode(self): @@ -125,9 +126,7 @@ def test_v0_4_examples_mention_compare_as_starter_pattern(self): self.assertIn("fastaguard compare", text) self.assertIn("starter", lower) self.assertIn("local", lower) - self.assertIn("WARN exits 1", text) - self.assertIn("FAIL exits 2", text) - self.assertIn("fail-fast starter", lower) + self.assertIn("gate.status", text) def test_v0_5_submission_readiness_docs_are_present(self): readme = self.read("README.md") diff --git a/tests/python/test_release_metadata.py b/tests/python/test_release_metadata.py index 76e9dd4..984cd20 100644 --- a/tests/python/test_release_metadata.py +++ b/tests/python/test_release_metadata.py @@ -10,10 +10,10 @@ class ReleaseMetadataTest(unittest.TestCase): - def test_package_targets_v0_5_0(self): + def test_package_targets_v0_6_0(self): cargo = tomllib.loads((ROOT / "Cargo.toml").read_text()) - self.assertEqual(cargo["package"]["version"], "0.5.0") + self.assertEqual(cargo["package"]["version"], "0.6.0") def test_bioconda_recipe_tracks_published_v0_5_0_archive(self): recipe = (ROOT / "packaging" / "bioconda" / "meta.yaml").read_text() @@ -53,6 +53,16 @@ def test_v0_5_0_release_notes_exist(self): self.assertIn("--gate submission", text) self.assertIn("--submission-target generic|ncbi", text) + def test_v0_6_0_release_notes_define_conventional_exit_contract(self): + notes = ROOT / "docs" / "releases" / "v0.6.0.md" + + self.assertTrue(notes.exists()) + text = notes.read_text() + self.assertIn("FastaGuard v0.6.0", text) + self.assertIn("Successful report generation exits with code `0`", text) + self.assertIn("`input_path`", text) + self.assertIn("`gate.status`", text) + def test_bioconda_recipe_has_publishable_v0_5_0_source_sha(self): recipe = (ROOT / "packaging" / "bioconda" / "meta.yaml").read_text() marker = "REPLACE" + "_WITH_" diff --git a/tests/schema_contract.rs b/tests/schema_contract.rs index e38d661..74629f4 100644 --- a/tests/schema_contract.rs +++ b/tests/schema_contract.rs @@ -271,7 +271,7 @@ fn freshly_generated_outlier_report_validates_against_json_schema() { .arg("--multiqc") .arg(&multiqc) .assert() - .code(1); + .success(); let schema = read_json(Path::new("schema/fastaguard.schema.json")); let report = read_json(&json);