Cap the heavy run so the pool stays fair

One long or expensive run in a shared pool can take the concurrency, the call budget or the connection slots that every other run depends on. The other runs do not fail for their own reasons. They wait behind a neighbour that never asked for the whole pool.

Give every run a bounded share

A share is a cap on the resource a single run may hold, enforced whether or not the pool is busy. The cap costs some idle capacity when the pool is otherwise free, and that is the trade: predictable latency for everyone, in exchange for not using every slot.

Separate the heavy work from the interactive path

Work with very different shapes should not share one queue. A batch run and a user-facing run have different tolerance for delay, so putting them in front of the same shared resource means the slower shape sets the latency of the faster one.

Find the neighbour by attribution, not by an average

Pool-wide averages hide the run that is taking everything. Per-run figures, cost, duration and calls attributed to the run identifier, are what make one heavy neighbour visible as a row rather than a suspicion.

def admit(run, pool, share):
    if pool.used(run["id"]) >= share:
        return {"admitted": False, "reason": "over_share"}
    if pool.free_slots() == 0:
        return {"admitted": False, "reason": "pool_full"}
    pool.reserve(run["id"])
    return {"admitted": True}

Expect the idle capacity

Isolation leaves some capacity unused. That is the price of a bound that holds even when the pool is empty, and a cap that only applies while others are waiting is not really a cap.