Changing Secrets and Prompts Without a Gap

Lesson 2 of 4

Version a Prompt

Estimated time: 10 minutes

Version a prompt like any other artifact

A prompt is an argument to a model, and a small edit to it can change behaviour as much as a code change. Edited in place with no record, it is the one artifact in the system nobody can audit after the fact.

Give every prompt a version

Keep the older versions rather than overwriting them. When behaviour changes and no code changed, the prompt is the first place to look, and the old text is what makes the comparison possible.

Record the version each run used

A result cannot be tied to the text that produced it unless the run stores the prompt version it used. Without that link, a behaviour change is a mystery.

def pin(prompt):
    return {"version": prompt.version, "text": prompt.text}

Change on purpose, roll back rather than edit forward

A new version should earn its place through a comparison, not by being typed into the live prompt. When one behaves badly, serve the known version again instead of editing forward while runs depend on it.