Spring Boot Under Load: Pools, Timeouts, and Backpressure

Almost every "the app got slow under load" incident I've debugged came down to one of three things, and none of them show up in development. In dev there's one user — you. Under real traffic, the app's limits start mattering, and Spring Boot's defaults are tuned for getting started, not for surviving a spike.

Here are the three that bite, in the order they usually bite.

1. The connection pool is your real concurrency limit

Spring Boot uses HikariCP, and its default maximum pool size is 10. That number, not your thread count, is often the true ceiling on how many requests can do database work at once. Request 11 doesn't fail — it waits for a connection, and to the user that looks identical to a slow database.

spring.datasource.hikari.maximum-pool-size=20
spring.datasource.hikari.connection-timeout=3000   # ms to wait for a connection
spring.datasource.hikari.max-lifetime=1800000       # recycle before the DB drops it

The instinct is to crank the pool to 100. Resist it. The pool should be smaller than what your database can handle — a Postgres tuned for 100 connections will thrash if every app instance opens 100. The right size is a function of your database's limits divided across your instances, not a number you wish for.

2. Every outbound call needs a timeout

The default timeout on many HTTP and database clients is infinite. That's fine until a downstream service gets slow — then your threads block on it, one by one, until the whole app is frozen waiting on someone else's outage. This is how a single slow dependency takes down a healthy service.

@Bean
RestClient restClient(RestClient.Builder builder) {
    var factory = new SimpleClientHttpRequestFactory();
    factory.setConnectTimeout(2000);   // give up connecting after 2s
    factory.setReadTimeout(5000);      // give up reading after 5s
    return builder.requestFactory(factory).build();
}

The rule is absolute: no call leaves the process without a timeout. Database queries, HTTP calls, cache lookups, queue operations — all of them. A request that fails in 5 seconds is recoverable. A request that hangs forever is an outage.

3. No backpressure means you fall over instead of degrading

When work arrives faster than you can process it, you have two choices: absorb it or reject it. Absorbing it — unbounded queues, unlimited threads — feels generous and ends in an out-of-memory crash. The healthier failure is to shed load: reject or queue with a limit, so the requests you do accept stay fast.

For heavy work, that means getting it off the request thread entirely and onto a bounded queue:

// Accept, enqueue, return immediately — don't do the slow work inline
public JobResponse submit(SubmitRequest req) {
    if (queue.size() >= MAX_DEPTH) {
        throw new TooManyRequestsException("Try again shortly.");
    }
    String jobId = queue.enqueue(req);   // Redis/Bull, RabbitMQ, etc.
    return new JobResponse(jobId, "queued");
}

A bounded queue that returns a clean 429 when full is doing its job. It protects the workers behind it and keeps the accepted requests responsive. An unbounded one just delays the crash and makes it bigger.

Actually seeing the limits

You can't tune what you can't see. Expose the metrics and watch them under load:

management.endpoints.web.exposure.include=health,metrics,prometheus

Hikari publishes hikaricp.connections.pending — the number of requests waiting for a connection. If that's ever above zero under normal traffic, your pool is the bottleneck, full stop. Watch pool usage, request latency percentiles (p99, not average), and queue depth.

The pattern behind all three

Every one of these is the same lesson: bound your resources and fail fast. An unbounded pool, an infinite timeout, and an unlimited queue all share the same failure mode — they defer the problem until it's catastrophic instead of surfacing it while it's still a fast, handleable error. Put limits everywhere, make hitting a limit a visible signal, and the app degrades gracefully instead of collapsing all at once.