Note to Self: HTTP Server KeepAlive Timeout

Note: I have ran into this problem on Apache httpd, but it could just as easily affect any other HTTP server.

The idea behind “KeepAlive” is straightforward: instead of opening a new connection for every request, the browser opens one connection to the server and reuses it to fetch many resources, such as the main page, stylesheets, JavaScript files, and images. In theory, this cuts page load times and gives visitors a better experience. So far, so good.

Apache httpd manages this feature with three directives, all part of the core module:

• KeepAlive: on or off (default: on)
• KeepAliveTimeout: a number of seconds (default: 5)
• MaxKeepAliveRequests: a positive integer (default: 100)

Apache httpd has no way to serve KeepAlive and non-KeepAlive requests side by side (for instance, allowing up to 100 KeepAlive requests per connection and handling anything beyond that differently). You have to decide on the behavior up front.

Time for some math. Starting from the MaxClients value (default: 256), how many new clients per second can the server handle without hurting the user experience? You might answer “MaxClients,” but let’s not jump to conclusions. A few factors come into play:

• The time between opening a connection and the KeepAliveTimeout expiring. With default settings that’s at least 5 seconds, and in typical real-world conditions probably 7 to 10 seconds.
• Fluctuations in traffic (spikes).
• Apache httpd’s own overhead, such as spawning new processes to handle incoming connections.

Things get complicated quickly. But if we assume a typical 7 seconds per client, during which a server process is either busy or waiting for data, and that all clients behave roughly the same way, then the default MaxClients of 256 supports a steady rate of 256 / 7 ≈ 36 new requests per second. Any spike above that will cause slow page loads and a poor user experience.

If turning KeepAlive off isn’t an option, plan around the worst spike you expect to handle (for example, 2x or 3x the average during your busiest period). If your busy-period average is 50 requests/second, the server should be able to cope with 100 or even 150 requests/second without degrading the user experience. If the hardware is already pushed to its limit, you’ll need other approaches, such as multiple servers or load balancers (which I won’t go into here).

For a peak of 150 requests/second with the default 5-second KeepAliveTimeout, you’d likely need to raise MaxClients to around 1,000. If you’re using the prefork MPM, ServerLimit needs to be raised as well.

Don’t rely on the defaults. Begin your parameter calculations with the outcome you want to achieve. On hardware that is short on memory, disabling KeepAlive is the best way to squeeze out extra performance. And don’t assume your hardware can deliver what your calculations demand; when something fails or was planned badly, treat it as a chance to learn.

That’s all for today. Have fun!