docs(tutorial): fix step 15 candidate-vs-baseline coherence#221
Open
placerda wants to merge 1 commit into
Open
docs(tutorial): fix step 15 candidate-vs-baseline coherence#221placerda wants to merge 1 commit into
placerda wants to merge 1 commit into
Conversation
Step 15 told readers to paste a byte-identical baseline as he fix, which made prompt_deploy stage report action: reused. The Release Evidence card then showed Target: travel-agent:2 unchanged, which felt contradictory because the tutorial never named the candidate-vs-baseline split. Updates: (1) mental-model gains a 4th bullet naming the split; (2) journey-table Fix and redeploy row updated; (3) step 15 prompt is now a small additive improvement (require currency codes) so stage reports action: created with a new candidate_agent version; (4) explanation block walks through foundry-agent.json, the Target vs candidate_agent fields, and where durable identity lives; (5) recording-note tells readers who already pushed a byte-identical restore to push one more commit; (6) success-criteria checklist matches the new step 15. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Step 15 of the prompt-agent quickstart instructed the reader to paste
back a byte-identical copy of the baseline prompt as "the fix". That
made
prompt_deploy stagereportaction: "reused"and keepcandidate_agent: travel-agent:2. When the reader then looked at theRelease Evidence card and saw
Target: travel-agent:2, the naturalreaction was "I just shipped a fix — why is nothing new here?".
The card was correct (the Release Evidence
Targetmirrors whatagentops.yamlpins — the declarative baseline), but the tutorialnever made the candidate-vs-pin distinction explicit, so the story
read as "the fix didn't take".
What this PR changes
Docs-only. No code, no schema, no behavior.
PRs create and evaluate a new candidate Foundry version; the
agentops.yamlagent:pin (and therefore the Release EvidenceTargetfield) only moves when you explicitly promote a baseline.to match the new step 15: push an improved prompt that restores the
guardrails and adds a small refinement, so a brand-new candidate
version is created (instead of a
reusedno-op).prompt: keeps all four original structural rules, adds one narrow
rule ("include currency code when known"). The change is small
enough that similarity scores against the existing dataset should
stay above thresholds, but real enough that
prompt_deploy stagereportsaction: "created"and a newcandidate_agent: travel-agent:N.new
foundry-agent.json, the candidate-vs-pin split, and wheredurable cross-environment identity lives (
prompt_sha256,git_sha).byte-identical restore in an earlier take, push one more commit
with the improved prompt and the stage step will report
action: "created"instead ofreused.prompt" bullet to mention the new candidate version and the
unchanged baseline pin, so the checklist matches step 15.
Why a narrow additive change for the new prompt
A purely-restorative "fix" can't show the
action: "created"+ new-candidate-version part of the story — bydefinition, byte-identical content reuses the existing version. We
need the fix to be a real content change. We also can't change the
prompt too much, or we risk dropping similarity scores below the
PR-gate thresholds and turning step 15 back into a failing run.
The chosen new rule (require currency code when monetary amounts are
mentioned) is narrow and additive, keeps the four structural
guardrails intact, and stays in scope for the dataset used by the
tutorial.
Release impact
None — this is a docs change. Will land on
developand ship with thenext release cut.
Test plan
table formatting still align.
the success-criteria checklist line was affected and is
updated.
Related context
Release Evidence
Target: travel-agent:2after the "fix" feltcontradictory.
foundry-agent.jsonartifact from
https://github.com/placerda/agentops-prompt-quickstart/actions/runs/26714275682
(
action: "reused",candidate_agent: travel-agent:2,prompt_sha256identical to baseline).