Rehearsing Failure and Expiring Old State
Lesson 3 of 4
Let Records Expire on a Clock
Estimated time: 10 minutes
Let records expire on a clock you control
State that never expires grows until it becomes the fault. But a record that vanishes sooner than its reader expects breaks work that depended on it, so the clock matters as much as the limit.
Set a time to live per record kind
One number for every kind of record is either too short for the slow reader or too long for the fast one. Set a time to live that matches how long something actually needs the record.
Expire on the system clock
An expiry decided by a reader is an expiry that can be argued with. Let the system hold the clock, so the boundary is the same for everyone.
def expired(record, now):
return (now - record.created) > record.ttl
Sweep on a schedule and watch the gap
Removal has to keep pace with arrival. Alert when the sweep falls behind the rate of new records, because that gap is how a store fills while nothing looks wrong.