Drain against a deadline

Restarting a worker in the middle of a write produces a record that is half written, and a retry that cannot tell whether the work happened. A drain avoids that with four moves in a fixed order: stop the intake, wait for work in flight, record what did not finish, restart and replay.

def drain(in_flight, deadline):
    while in_flight() and not deadline.expired():
        sleep()
    return {"unfinished": in_flight(), "deadline_hit": deadline.expired()}

The deadline is what keeps the drain bounded. Without it a single stuck task holds the restart open indefinitely, which turns a routine deploy into an incident.

New work is not blocked. Hold it in the queue or route it to a healthy worker; draining empties one worker, it does not stop the system.