Moving State and Reserving Capacity

Lesson 2 of 4

Size the Pool Before It Runs Dry

Estimated time: 10 minutes

An empty pool looks like a slow backend

A connection pool is a fixed set of reusable links. Every worker that holds one while it waits on a slow call takes it out of circulation, and once enough workers do this the pool empties. New work then waits for a link, not for a server, so the delay looks like the backend is slow when the backend has not been called at all.

Watch the waiters, not just the errors

The early signal is the number of waiters and how long the longest has waited. By the time calls are failing the pool has been empty for a while.

Set a hold limit

A hold limit shorter than the slowest call reclaims links that a stuck worker would otherwise keep for the whole run. Return every link in a finally step so one error does not leak one for good.

def checkout(pool, hold_limit):
    conn = pool.acquire(timeout=hold_limit)
    if conn is None:
        return None  # report a waiter rather than block the turn
    return conn

Size above peak

Measure the highest number of links in use, then set the pool above it with headroom. Sizing to the average leaves the pool empty at exactly the moment it matters.