Iterating a prompt without losing what worked
f you rewrite a prompt from scratch every time it misbehaves, you’re throwing away working behavior along with the bug. Treat prompts like code instead: change them in small, visible diffs and rerun the same quick tests, so improvements stack instead of reset.
Prompt Iteration Loop
- Baseline promptchange
- One-sentence diffrerun
- Rerun mini testscheck
- Spot regressionsdecide
- Evolve or forknext
Article mapOpen the visual summary
Prompt Iteration Loop
- Baseline promptchange
- One-sentence diffrerun
- Rerun mini testscheck
- Spot regressionsdecide
- Evolve or forknext
Table of Contents10 sections
- What you’ll be able to do after this· 1 min
- Where you’re starting: structured prompts, fragile workflow· 1 min
- Why “rewrite from scratch” keeps breaking good behavior· 1 min
- A simple versioning setup: flat files, names, and dates· 1 min
- Your first iteration log: baseline, change, re-run· 1 min
- Diff-based edits: only change one thing at a time· 1 min
- Eval-by-eye: a 3-input mini test set you always rerun· 1 min
- Fork vs evolve: one prompt or a new branch?· 1 min
- Spotting regressions early (and what to do next)· 1 min
- Cheatsheet: your personal prompt-iteration loop· 1 min
What you’ll be able to do after this
- Keep prompts in simple, named files so you can revisit and improve them instead of rebuilding from memory.
- Make tiny, focused edits and see clearly when a change helped, hurt, or did nothing.
- Run a 3-input mini eval after each change so regressions show up immediately, not a week later in production.
- Decide when to fork a prompt into a new variant versus evolve the existing one.
Where you’re starting: structured prompts, fragile workflow
You’re probably already past the “ask me anything” stage. You write prompts with sections, maybe something like:
You are a helpful assistant for my SaaS.
TASK: Draft a reply to this customer email.
CONSTRAINTS:
- Match the product’s friendly-but-direct tone.
- Keep it under 150 words.
- Ask exactly one follow-up question.
The problem: when this starts failing on some edge case, you nuke it and retype from scratch. Some things get better, others mysteriously break, and you’re left with a feeling that last month’s version handled certain cases better.
This article assumes you can already write a structured prompt. What you don’t have yet is a prompt-iteration habit: versioned text, small diffs, and a fast way to see whether a change actually helped.
Why “rewrite from scratch” keeps breaking good behavior
- Rewrite from scratch
- Change dozens of variables
- New version is worse
- No idea which change
Models like Claude Opus 4.7 or GPT-5.1 are non-deterministic and incredibly sensitive to wording, order, and examples. You already know this; you’ve seen a single extra line flip the behavior.
When you rewrite from scratch, you change dozens of variables at once: role, order, constraints, examples, even whitespace. If the new version is worse, you have no idea which change did it.
Think of it like refactoring a backend service by deleting it and re-implementing from memory. You don’t just lose bugs; you lose the half-dozen weird edge cases the old version accidentally handled well.
Treat every prompt change like a code diff: explicit, small, and reversible. The goal is not a “perfect prompt” you’ll never touch again. The goal is a history you can read, reason about, and roll back when a well-meaning tweak quietly breaks a case that used to work. You’re not casting spells; you’re editing a contract with a probabilistic system.
The fix is boring and powerful: keep the text stable, change one thing at a time, and re-test the same few inputs every time you edit.
A simple versioning setup: flat files, names, and dates
You don’t need a SaaS dashboard for this. A folder and some text files are enough.
Use one file per reusable prompt. Inside the file, keep a short header and dated versions.
# prompt: support_reply
# purpose: Reply to customer emails for ACME SaaS
## 2026-05-05
[version text here]
## 2026-05-08
[revised version text here]
Keep filenames flat and boring:
support_reply.prompt.mdblog_outline.prompt.mdsql_explainer.prompt.mdThe name should answer: “What job does this prompt do?” The date marks snapshots inside the file. That’s enough to:
If you’re comfortable with Git, put this folder in a repo. If not, don’t worry. The main thing is: prompts live in files you control, not just chat history.
Your first iteration log: baseline, change, re-run
Let’s walk a full loop with something real. Set aside 20 minutes and actually do this.
- Pick one prompt you reuse often.
- Example: your support-reply prompt or “summarize this meeting” prompt.
- Create a file like
support_reply.prompt.mdand paste the current version under a## 2026-05-10header. - Define three concrete test inputs that represent real work:
A straightforward case.A tricky or angry case.A weird edge case you sometimes see.
- Run all three through the existing prompt in your usual model (e.g., Claude Opus) and paste the outputs under each input in the same file.
You’ve just created your first mini iteration log: one snapshot of the prompt, three baseline outputs.
Now pick one thing you wish this prompt did better. For example: “Ask a single clarifying question, every time, near the end.” We’ll use that in the next section to do your first disciplined change.
Diff-based edits: only change one thing at a time
The fastest way to stop breaking prompts is to constrain each edit so you can describe it in one sentence.
For the support-reply example, you might go from this:
CONSTRAINTS:
- Match the product’s friendly-but-direct tone.
- Keep it under 150 words.
- Ask exactly one follow-up question.
to this:
CONSTRAINTS:
- Match the product’s friendly-but-direct tone.
- Keep it under 150 words.
- Ask exactly one follow-up question, placed in the final sentence.
That’s a tiny diff. You didn’t touch the role, task, or examples; only clarified where the question should appear.
Immediately below your ## 2026-05-10 version, add:
## 2026-05-10b
# change: Force follow-up question into final sentence.
[updated prompt text]
That # change: line is your one-sentence commit message. If you can’t summarize it cleanly, you edited too much.
Now re-run the same three inputs with the new version and paste the outputs under a new heading like ### Outputs 2026-05-10b in your file.
Eval-by-eye: a 3-input mini test set you always rerun
You just did a very small eval. No frameworks, no dashboards—just eyeballs and a notebook.
For each of your three inputs, compare old vs new outputs:
You want to label each input with a short verdict:
Input A: Better – question now always at the end, everything else stable.
Input B: ⚠️ Mixed – question is at the end but tone feels too sharp.
Input C: Worse – blew past 150 words.
This is eval-by-eye. It’s subjective, but it’s still structured. Over time, you’ll know that “2026-05-10b is better on A/B, worse on C.” That’s already miles better than “the new version feels off.”
If a change clearly helps all three, keep it. If it obviously hurts one case, either adjust and re-test or roll back that specific wording.
Fork vs evolve: one prompt or a new branch?
Evolve same prompt
- Same overall task, minor variation
- New constraint fits naturally
- one prompt
- ordinary emails
- support_reply.prompt.md
Fork into new prompt
- Different users or risk level
- Adding constraint makes existing ones contradictory
- new file
- high-risk refund cases
- support_refund_sensitive.prompt.md
At some point, your prompt starts carrying conflicting jobs. Maybe your support prompt is handling:
Trying to make one prompt handle all of these with a single set of examples is how you end up with a 300-line monster that’s hard to reason about.
Use this table when you’re unsure whether to keep evolving one prompt or fork it into a new file:
| Situation | Evolve same prompt | Fork into new prompt |
|---|---|---|
| Same overall task, minor variation (e.g., tone, length) | ||
| Different users or risk level (internal note vs user-facing reply) | ||
| New constraint fits naturally in existing structure | ||
| Adding constraint makes existing ones contradictory | ||
| Needs different few-shot examples entirely |
Example: you might keep support_reply.prompt.md for ordinary emails, and create support_refund_sensitive.prompt.md for high-risk refund cases with stricter tone and escalation rules.
When you fork, treat it like a new repo: copy the last good version plus three fresh eval inputs tailored to the new scope.
Spotting regressions early (and what to do next)
A regression is simple: something that used to work, stops working after your change.
In prompt-land, that might look like:
When you run your three-input mini eval and see a regression:
- 1Confirm it’s real. Re-run that one case with the same prompt and a low temperature. If it still fails, it’s on the prompt, not randomness.
- 2Blame the last change, specifically. Look at your
# change:note. Did you alter constraints, examples, or order? - 3Roll back locally. Undo just that part of the edit and re-test.
- 4Try a smaller move. If you changed two constraints, change one; if you rewrote the whole intro, tweak a single sentence.
The mindset is the same as debugging code: assume the last diff is guilty until proven innocent.
Cheatsheet: your personal prompt-iteration loop
By this point you’ve:
This is the loop you’ll reuse for every “serious” prompt you care about:
Run this loop a few times a week and you’ll accidentally build a personal prompt library that gets sharper over time instead of resetting every time a reply annoys you.
Field reference: iterating prompts without regressions
Prompt naming convention
Use flat, job-focused names so you can guess a prompt’s purpose from the filename alone. Pattern: <area>_<job>.prompt.md (e.g., support_reply.prompt.md, sales_email.prompt.md, meeting_summary.prompt.md). Inside each file, use dated headers like ## 2026-05-10 for versions and a one-line # change: summary. Keep one primary prompt per file; forks become new files rather than buried sections.
Three-input mini eval set
For every reusable prompt, define exactly three baseline inputs: (1) a typical, boring case you see often, (2) a hard or emotional case, and (3) a weird edge case that has bitten you before. Write them as concrete text snippets you can paste verbatim. After any non-trivial change, re-run these three and mark each result as better/same/worse with one sentence on why. If all three are ‘better or same’, the change is usually safe to keep; if one is worse, either adjust or roll back.
⚡ Diff-then-test loop
Before editing, write a one-sentence goal (“Make JSON always valid” or “Question always at the end”). Change the smallest possible part of the prompt that could achieve that goal—one constraint line, one example, or a short instruction, not the entire block. Save as a new dated section, note # change: ..., then immediately run your 3-input eval. If you can’t point to a specific line that caused a regression, the diff was too big; revert and redo as two smaller edits.
⚠️ Regression signals checklist
Watch for four specific red flags after each change: (1) constraints that used to be obeyed now fail (length, format, tone), (2) output structure becomes inconsistent between runs, (3) previously correct edge cases now behave oddly, and (4) the model starts ignoring your tools or schema. If any appear on your eval cases, do a targeted rollback: undo just the last wording change around that behavior and re-test. Don’t accept ‘it’s probably fine’—either confirm it’s fixed or revert the version.
Want a more guided way to practice this?
FAQ: making prompt edits compound instead of regress
️ How do I version prompts without a tool?
Put your prompts in plain-text files and let the file system be your first version-control system. Use one file per prompt job (for example, support_reply.prompt.md) and create dated sections inside the file like ## 2026-05-10. Under each date, store the full prompt text, a one-line # change: summary, and, ideally, the three test inputs and outputs you used. If you’re comfortable with Git, put the whole folder under source control so you get real diffs and history; if not, the dated headers already give you a readable, low-friction version log.
⚖️ When do I know an iteration made it worse?
An iteration is worse when something that used to work reliably now fails on the same input with the same model settings. The clean way to detect that is to re-run your fixed three-input eval set after every non-trivial change and compare old vs new outputs side by side. If any case that used to be ‘good enough’ is now clearly worse, breaking a constraint, changing tone, or losing structure, you treat that as a regression. At that point you either tweak the specific changed line that caused it or roll back that version entirely and try a smaller edit.
Should I keep old prompt versions?
Yes, at least until a new version has survived a few rounds of use without surprising failures. Old versions are your safety net when a ‘better’ prompt quietly breaks an edge case you don’t see every day. Keeping the last two or three dated versions in the same file is usually enough; you don’t need to preserve every micro-change forever. Over time, you can prune obviously-bad versions and keep only the ones you might reasonably roll back to, much like you might keep a few known-good releases of an API service.
What is a good test set of inputs?
For a single-person workflow, three to five carefully chosen inputs beat a huge, sloppy list. Start with: one common case that represents the majority of your usage, one difficult or emotionally-loaded case where nuance matters, and one oddball that’s broken things before (long text, tricky formatting, multilingual, etc.). Each input should be realistic, not synthetic, ideally copy-pasted from real work with private details anonymized. As you see new failure modes in daily use, consider swapping one of your eval inputs for a fresher, more representative one so your tests stay aligned with reality.
⏱️ How often should I iterate a prompt that already ‘works’?
Not every run needs a new version. If a prompt is meeting your bar on real tasks and your three eval cases stay green, leave it alone and spend your time elsewhere. Iterate when you see recurring friction: you keep fixing the same issue by hand, the model ignores a constraint, or collaborators misuse the prompt. When you do decide to change it, treat that change as a small, logged experiment and re-run your eval set. A handful of focused iterations per month on high-impact prompts is usually enough to keep them sharp without constant churn.
Bringing it together: prompts as living source, not disposable text
You don’t need a lab, a dashboard, or an eval team to stop your prompts from regressing. You just need to stop treating them as throwaway text and start treating them as living source with history.
Keep each serious prompt in a named file. Make one small, intentional change at a time. Re-run the same three test inputs and write down what got better or worse. Roll back when needed.
Do that for a few weeks and you’ll have something most teams don’t: a compact, readable library of prompts that actually get better with age. Not because you found the magic template, but because you built a simple, boring habit of editing them like code.
