A new architecture for HTTP/2 in Haskell

GHC 9.6 provides listThreads finally. Just for curious, I have implemented a thread monitor in http2-server, a test command tool for the http2 library in Haskell. This revealed that huge numbers of Haskell lightweight threads are used. See the following picture of servers from Experience Report: Developing High Performance HTTP/2 Server in Haskell.

The old architecture makes use of the worker pool. Workers of a fixed number are spawned by the worker manager in advance. A worker takes an HTTP request from the input queue, works for the request, generates an HTTP response and then enqueues the response to the output queue.

Then sender dequeues an HTTP response, fills the output buffer with the available data of the response, flushes the output buffer if necessary, and enqueue the response to the output queue again if it has more data. If the flow control window in the stream level is closed for the HTTP/2 stream, it spawns a waiter thread. The waiter thread wait until the window gets open then enqueues the response again.

For the response of the streaming type, the sender also checks if the streaming data is available. If not, the sender spawns a waiter. The thread monitor revealed that the number of waiters is much larger than I expected. This ruins the saving number of worker threads.

To avoid the thread number explosion, I need to give up the worker pool. Instead, a worker is spawn for an HTTP/2 stream on demand. The sender pushes a response back to the corresponding worker. The worker itself takes care of the flow control window in the stream level or the availability of streaming data.

This new architecture simplified the code drastically:

• The worker manager is not necessary anymore.
• The code to go to the next request safely is removed from the worker.
• The sender should only take care of the flow control window in the connection level.

Version 5.3.0 or later of the http2 library provides this new architecture.

Changes to the paper in Haskell Symposium 2016

• Client and server code was extracted from Warp to the http2 library. See "HTTP/2 server library in Haskell" and "Implementing HTTP/3 in Haskell" for more information.
• Section 3 "Priority" is outdated because the priority feature was removed from the HTTP/2 specification. To avoid vulnerability, the priority code was eliminated from the http library.
• Section 4 "HTTP/2 Implementation in Warp" is also outdated as described in this particle. In particular, Section 4.1 "Optimistic Enqueueing" is meaningless since the number of workers is controlled by SETTINGS_MAX_CONCURRENT_STREAMS (maxConcurrentStreams, default: 64).
• In Figure 7 and Figure 8 in Section 5 "Evaluation", the stack of Haskell threads consumes large memory. This is because the default value of the stack chunks in GHC (32KiB) is not kind. With this settings, the stack grows like 1KiB, 33KiB, 65KiB, etc. 33KiB is too large as the next step for server usage. Recently, I specify -kc2k to my project code where the step is 1KiB, 3KiB, 5KiB, etc.

I'm a little bit disappointed since I should admit that it's hard for me to write a solid paper about software.

Acknowledgment

I thank Edsko de Vries and Finley McIlwaine for testing the new architecture. They are big contributors to the http2 library in Haskell.