What Is a Connection Pool?

Why pools exist

Opening a database connection is expensive — a TCP handshake, authentication, session setup — and can easily cost more than the query that follows. A pool pays that cost once, keeps the connections open, and lends them out to requests that need one.

The pool is deliberately small, and that is the point. Databases handle a limited number of concurrent connections efficiently and get slower beyond it, so the pool is a throttle protecting the database as much as a cache of sockets.

The classic load test bottleneck

Connection pools produce the most frequently misdiagnosed result in load testing: response time climbing steadily while CPU sits near idle on every tier. Nothing is working hard, and everything is slow.

The cause is queueing rather than computing. With a pool of 20 and 200 concurrent requests, 180 requests are waiting for a connection, and that wait is recorded as response time. Adding application servers makes it worse, because each new instance opens its own pool against the same database — the total is pool size × instance count, and it is easy to exceed the database’s own connection limit that way.

Diagnosing pool exhaustion

The signature is distinctive once you know it, and it appears in both stress and soak tests — the latter usually because connections are leaking rather than because the pool is too small.

• Response time rises roughly linearly with concurrency while CPU stays flat.
• Pool wait-time or checkout-time metrics climb, if the framework exposes them.
• Errors, when they come, are pool-timeout messages rather than query errors.
• In a soak test, a pool that never returns to idle between requests points at connections that are not being released.

Sizing a pool

Bigger is not better, and this surprises people. Past the point where the database can execute queries in parallel, extra connections add context switching and lock contention rather than throughput — a smaller pool with requests queueing briefly in the application often beats a large one with queries queueing inside the database.

The constraint that actually matters is the total across every client: pool size × application instances, plus background workers, migrations, and admin tools, must stay under the database’s own connection limit. Autoscaling makes this easy to get wrong, because the total moves whenever the instance count does.