By Brett Segraves
In the world of the web there are multiple methods of connecting to a web server. The primary methods are HTTP 1.0, HTTP 1.1, and HTTP 2.0. Of these three, the latter two use KeepAlive by default. What is KeepAlive? KeepAlive is when the server and client persist a connection for more requests to be made. Let’s say we use a web browser to connect to a website. If KeepAlive is enabled the server and client will establish a connection for the first request. Then, the connection will stay open for any future requests the client wishes to make. This can help drastically reduce the time needed for a request to be handled and served.
In Apache2, KeepAlive is enabled by default, with a KeepAliveTimout of 5. This means that after five seconds the persistent connection is closed, and a new connection must be made. This also means that concurrent connections are being used. All this means is that there are multiple clients connected at the same time to the server; it can be limited using the MaxClients directive.
Another thing to note is how apache2 listens for connections. Apache2 uses “Worker” processes, with one or more threads per worker. The idea of this experiment is to flood the Workers with keepalive connections, keeping them open in such a way that the listening threads become blocked. Due to hardware constraints, this experiment was performed using a local docker image as the apache server on a Core i7-6700 processor.
By opening and maintaining large numbers of KeepAlive requests, the listening threads of the apache server will be blocked while using minimal processing power and network bandwidth.
To perform these tests, I used the “httpd” docker image from docker hub and used the keepalive settings “KeepAlive On” and “KeepAliveTimout 5”. For the connections I used two python scripts, the blocking script spawned a number of threads that continuously connected and reconnected to the web server while the timing script only made a single connection but timed the request. The timing script was run on a separate system from the blocking script; however, it was on the same local network as the server.
The Blocking script used sockets to connect to the server, this was so that custom http requests could be made. However, it turned out that when a keepalive request is made to the server, instead of blocking one of the listener threads, it would answer the request and keep it open, but would also answer other requests in the same slot. As such, using a keepalive request yielded little result. Instead, the trick was to open a connection to the server as if to make a request but send nothing. By doing this I was able to lock many of the listeners in a “reading request” state. This, however, also didn’t yield the kind of results I’d expected.
Using the environment described above, two tests were performed. The timing script was run both without the blocking, and with. As shown below, there is a clear effect. Both the mean and standard deviation of the test with interference is significantly higher.
Another interesting thing to note is the number of reconnects per minute to the server. (From the blocking script) After eight minutes, the number of reconnects levels off to 1200 reconnects per minute.
With this in mind, I feel it’s hard to say whether or not system limitations are to blame, or if it’s just apache being good at connection management. Also note that the requests were made to the default index page, meaning there wasn’t much data to be transferred even if a proper request was being made. Keep in mind each reconnect need only consist of a TCP handshake.
It’s clear that making connections without proper requests has a negative impact on the performance of the server but making keepalive requests to block listeners simply didn’t work. It may be possible, with a different environment, to bring the server to a crawl using blank connections. Thus, completely locking up the server with low network bandwidth and low processing usage seems like it isn’t fully impossible given that the results above are from a single system running the blocking script.