Running a change against real requests first

A version can clear every check and still stumble on the first request from a real user, because the checks never carried the shape of live input. Letting the new version reply to copies of real requests, while the version in service keeps serving, surfaces that gap while nobody depends on the reply.

Send the same request to both versions

Each real request goes to both versions. Only the version in service answers the user, and the other writes its reply to a record. A poor reply from the new version reaches nobody, so the cost of finding it is a comparison rather than an incident.

Compare the two replies on one request

Because both replies describe the same request, any difference is a difference in behaviour rather than a difference in load. Split them into differences of wording and differences of meaning, and stop the change only for the second kind.

def compare(serving_reply, trial_reply):
    agreed = serving_reply["value"] == trial_reply["value"]
    return {"agreed": agreed, "serving": serving_reply["value"], "trial": trial_reply["value"]}

Agreement covers only what was seen

The two versions agree on the requests the trial version has actually met. A path that receives no copies has not been tried, so either send it traffic or leave it where it is and say so.

Hold the old version close

Keep the previous version ready until the new one has answered real requests for a while. A change that cannot be undone in one move was decided before the evidence arrived.