Skip to content

apex_sim: apex-finder simulator app + sensitivity bench#67

Merged
jspaezp merged 1 commit into
mainfrom
feat/apex-sim
Jul 18, 2026
Merged

apex_sim: apex-finder simulator app + sensitivity bench#67
jspaezp merged 1 commit into
mainfrom
feat/apex-sim

Conversation

@jspaezp

@jspaezp jspaezp commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

New crate rust/apex_sim: an eframe/egui desktop app plus a bench for probing
the timsseek apex finder under synthetic conditions.

What it does

  • Synthesizes extraction products (chromatograms) with tunable knobs:
    peak shape, per-fragment noise, theoretical-vs-observed ratio mismatch
    (obs_scale), absent/empty transitions, randomly injected peak-like objects,
    precursor isotopes, global noise floor. Deterministic (seeded ChaCha8Rng).
  • Feeds them to the REAL scorer (timsseek TraceScorer / Extraction / traces).
    No scoring is reimplemented; the app runs the two production passes
    (compute_traces -> suggest_apex -> score_at).
  • UI: overlaid chromatograms (colored by ion series), transitions-x-cycles
    heatmap, and all intermediate traces (cosine/scribe/lazyscore/log_int/
    ms1/apex_profile) with TRUE / pass1 / pass2 apex markers, plus a full
    ApexScore decomposition (base_score, au/cg terms, 11 features).
  • Prints the final SimParams on window close for reproducibility.

Bench (examples/bench.rs, apex_sim::bench)

  • Runs a catalog of canonical scenarios (clean, moderate_noise,
    high_noise+interference, heavy_interference, mismatched_library,
    absent_top_fragment) over many seeds and prints per-scenario detail plus
    a comparison summary table (pass1/pass2 sensitivity, median error, us/run).
  • Simulation is built up front so it stays OUT of the scoring timer; a single
    TraceScorer is reused across runs (matches production ScoringWorker).

Fidelity

  • The sim applies the same filter_zero_intensity_ions +
    select_top_n_fragments(TOP_N_FRAGMENTS) the pipeline runs before scoring,
    so it scores exactly what timsseek would. Those three items are widened to
    pub in timsseek (only visibility, no behavior change).

Build / CI

  • egui/eframe/egui_plot gated behind a default-on 'gui' feature; the bench
    builds lean with --no-default-features (no egui/winit in CI).
  • .github/workflows/apex_bench.yml runs the bench (--no-default-features) on
    PRs touching apex_sim/timsseek and upserts a sticky results comment.

Validation

  • cargo test -p apex_sim (7 pass), and under --no-default-features (7 pass).
  • cargo build -p apex_sim (gui) and --no-default-features both clean.
  • Ran the bench locally; example sticky-comment output is the summary table.
  • Reviewed by a second pass (senior-eng); the two fidelity/methodology
    findings (production filtering + reused-scorer timing) are applied here.

New crate rust/apex_sim: eframe app that synthesizes extraction products
(tunable noise, ratio mismatch, absent transitions, injected peaks) and
feeds them to the REAL timsseek TraceScorer -- no reimplemented scoring.
Displays chromatograms, heatmap, and every intermediate trace with both
scoring passes' apexes plus a full score decomposition.

Bench (examples/bench.rs, apex_sim::bench): runs a catalog of canonical
scenarios (clean/noise/interference/mismatched-library/absent-fragment)
over many seeds, measuring apex-recovery sensitivity + scoring throughput.
Simulation is built up front so it stays out of the scoring timer; one
reused TraceScorer mirrors production's ScoringWorker.

Fidelity: sim applies the same filter_zero_intensity_ions +
select_top_n_fragments(TOP_N_FRAGMENTS) the pipeline runs before scoring
(those three widened to pub in timsseek).

GUI stack (egui/eframe/egui_plot) gated behind default-on 'gui' feature
so the bench builds lean under --no-default-features.

CI: .github/workflows/apex_bench.yml runs the bench --no-default-features
and posts a sticky results comment on the PR.
@github-actions

Copy link
Copy Markdown

Apex-finder bench

cargo run -p apex_sim --release --example bench -- 1000 2

Sensitivity + timing across canonical scenarios
=== clean ===
  config: n_cycles=245 sigma=1 noise_floor=0.02 random_peaks=200 true_apex=30
  runs=1000 tol=±2 cycles
  build :    38.32 ms total  (38.32 us/run)
  score :    29.62 ms total  (29.62 us/run, 33764 runs/s)
  scorer errors:    0
  pass1 sensitivity: 100.00%  (1000/1000)
  pass2 sensitivity: 100.00%  (1000/1000)
  pass2 |apex-truth| cycles: mean=0.00 median=0 max=0
  pass2 err histogram: 0:1000 1:0 2:0 3:0 4:0 5:0 6:0 7+:0 

=== moderate_noise ===
  config: n_cycles=245 sigma=1 noise_floor=0.2 random_peaks=50 true_apex=30
  runs=1000 tol=±2 cycles
  build :   163.51 ms total  (163.51 us/run)
  score :    28.61 ms total  (28.61 us/run, 34953 runs/s)
  scorer errors:    0
  pass1 sensitivity:  78.30%  (783/1000)
  pass2 sensitivity:  77.80%  (778/1000)
  pass2 |apex-truth| cycles: mean=21.12 median=1 max=214
  pass2 err histogram: 0:474 1:278 2:26 3:7 4:6 5:3 6:2 7+:204 

=== high_noise+interference ===
  config: n_cycles=245 sigma=1 noise_floor=0.5 random_peaks=100 true_apex=30
  runs=1000 tol=±2 cycles
  build :   292.95 ms total  (292.95 us/run)
  score :    28.46 ms total  (28.46 us/run, 35136 runs/s)
  scorer errors:    0
  pass1 sensitivity:  47.80%  (478/1000)
  pass2 sensitivity:  47.40%  (474/1000)
  pass2 |apex-truth| cycles: mean=48.45 median=7 max=214
  pass2 err histogram: 0:294 1:165 2:15 3:9 4:4 5:5 6:4 7+:504 

=== heavy_interference ===
  config: n_cycles=245 sigma=1 noise_floor=0.5 random_peaks=300 true_apex=30
  runs=1000 tol=±2 cycles
  build :   819.97 ms total  (819.97 us/run)
  score :    28.52 ms total  (28.52 us/run, 35062 runs/s)
  scorer errors:    0
  pass1 sensitivity:   8.40%  (84/1000)
  pass2 sensitivity:   8.40%  (84/1000)
  pass2 |apex-truth| cycles: mean=85.54 median=77 max=214
  pass2 err histogram: 0:53 1:28 2:3 3:4 4:11 5:8 6:8 7+:885 

=== mismatched_library ===
  config: n_cycles=245 sigma=1 noise_floor=0.5 random_peaks=100 true_apex=30
  runs=1000 tol=±2 cycles
  build :   292.88 ms total  (292.88 us/run)
  score :    28.59 ms total  (28.59 us/run, 34975 runs/s)
  scorer errors:    0
  pass1 sensitivity:  25.30%  (253/1000)
  pass2 sensitivity:  25.00%  (250/1000)
  pass2 |apex-truth| cycles: mean=71.34 median=51 max=214
  pass2 err histogram: 0:114 1:112 2:24 3:5 4:2 5:8 6:8 7+:727 

=== absent_top_fragment ===
  config: n_cycles=245 sigma=1 noise_floor=0.5 random_peaks=100 true_apex=30
  runs=1000 tol=±2 cycles
  build :   293.28 ms total  (293.28 us/run)
  score :    28.53 ms total  (28.53 us/run, 35052 runs/s)
  scorer errors:    0
  pass1 sensitivity:  10.40%  (104/1000)
  pass2 sensitivity:  10.30%  (103/1000)
  pass2 |apex-truth| cycles: mean=84.59 median=76 max=214
  pass2 err histogram: 0:24 1:65 2:14 3:8 4:8 5:8 6:8 7+:865 

=== summary (n=1000, tol=±2 cycles) ===
  scenario                    pass2%  pass1%   medErr    us/run
  clean                        100.0   100.0        0     29.62
  moderate_noise                77.8    78.3        1     28.61
  high_noise+interference       47.4    47.8        7     28.46
  heavy_interference             8.4     8.4       77     28.52
  mismatched_library            25.0    25.3       51     28.59
  absent_top_fragment           10.3    10.4       76     28.53

commit d445179

@jspaezp
jspaezp merged commit 3d98583 into main Jul 18, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant