Send one spare copy for the slow few

Most calls to a tool come back quickly and a few stall. A caller that waits on each call in turn pays the whole stall, and one stuck call can hold a step that had plenty of time left.

Send a spare after a measured delay

The spare copy goes to a different route, and only after the first copy has been slow for a chosen delay. The delay matters most: set it above the usual response time so ordinary calls are never doubled, and only the real tail pays for a second copy.

def with_spare(call, delay, cap, running):
    first = call("a")
    if still_waiting(first, delay) and running() < cap:
        return pick_first(first, call("b"))
    return first

Handle the copy that loses

Two copies mean the work and the spend can be counted twice unless the losing copy is cancelled or ignored. Take the first answer and let the other go.

Cap the extra share

A limit on how much extra work the spares may add keeps a sick route from turning one slow call into a storm of duplicates. Do not send spares for calls that are already known to be slow on every route.