feat: lead /compare with OpenEMR head-to-head, keep MockMed as reproducible anchor#143
Draft
abrichr wants to merge 4 commits into
Draft
feat: lead /compare with OpenEMR head-to-head, keep MockMed as reproducible anchor#143abrichr wants to merge 4 commits into
abrichr wants to merge 4 commits into
Conversation
…ucible anchor Replace the single MockMed stat block in the computer-use-agents section with two panels: the OpenEMR public-demo head-to-head (both arms 100%, 1.8x median latency, $0 vs $0.55/run at list, 0 vs ~24 model calls/run) as the primary result with its caveats disclosed inline, and the MockMed benchmark beneath it as the CI-reproducible anchor. Numbers from openadapt-flow benchmark/openemr/results.json (2026-07-08). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ
✅ Deploy Preview for cosmic-klepon-3c693c ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
One of the 20 compiled runs self-flagged postcondition drift on the final step and was verified saved by the arm-independent OCR check; add one sentence to the panel's caveat footnote so the page matches the canonical BENCHMARK.md disclosure. Headline numbers unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ
Compiled-first with agent fallback on detected halt: 20/20 at ~1/8 the agent's cost per successful run under injected drift, with caveats (synthetic detected-halt drift, assumed drift mix) noted inline and a link to the hybrid benchmark methodology. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ
Lead the page with the instrument thesis: self-healing GUI replay can silently write to the wrong record under UI drift, nobody measures it, we red-teamed our own engine five times (frozen corpus, FA 0.000% / ~26-28% safe-halt cost) and found two shipping competitors each 3/3 silent wrong-patient under identity drift. Add the honest DOM-selector exception. Benchmarks (OpenEMR/MockMed/ hybrid) reframed as the supporting efficiency case, not the lead. Prominent honest-limitations link to LIMITS.md plus the adversary log and competitor study. Build green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Updates the "We measured it" panel on /compare to lead with the OpenEMR head-to-head (real third-party EMR) and keeps MockMed beneath it as the CI-reproducible anchor.
Numbers are from
openadapt-flow/benchmark/openemr/results.json(generated 2026-07-08).Dependencies
The primary panel links to
benchmark/openemr/BENCHMARK.mdon openadapt-flowmain; that file lands with OpenAdaptAI/openadapt-flow#11 (awaiting review). Do not merge this PR before that one.Test plan
🤖 Generated with Claude Code
https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ