Mew.org is now speaking HTTP/3 (HTTP/2 over QUIC). If you gain access to the site using Firefox Nightly, the first connection would be HTTP/2 then the following connections should be HTTP/3 led by Alt-Svc:
.
This article explains insights which I found through the implementation activities of QUIC and HTTP/3 in Haskell.
HTTP/2 server library
I started implementing QUIC in January 2019. It took four months to reach a toy QUIC client since the negotiation part is really complicated. When I tackled the server side, my brain got befuddled. I have no idea on server architecture.
So, I got back to HTTP/2. As described in HTTP/2 server library in Haskell, I succeed to extract HTTP/2 server library from our HTTP/2 server.
QUIC client and server
After resuming QUIC implementation, I spent countless hours to develop QUIC. And finally, I joined 16th interop test event and 17th interop test event.
As described in Implementation status of QUIC in Haskell, I defined the following API:
runQUICServer :: ServerConfig -> (Connection -> IO ()) -> IO ()
When a QUIC connection is created at the server side, a designated lightweight thread is spawned with the Connection
type. This abstraction seems reasonable because Connection
hides internal information about a QUIC connection. However, I have no idea on how to abstract QUIC streams at that moment. So, I defined the following APIs temporally:
type StreamID = Int type Fin = Bool recvStream :: Connection -> IO (StreamID, ByteString) sendStream :: Connection -> StreamID -> Fin -> ByteString -> IO () shutdownStream :: Connection -> StreamID -> IO ()
These APIs seem awkward since it exposes stream identifiers which applications should not know. Anyway, through this development I got an insight that a lot of code can be shared between client and server.
HTTP/2 client library
Now I wanted to verify that HTTP/2 client library can be achieved by sharing a lot of server code. The result is promising. HTTP/2 library in Haskell now provides both client and server side and implements self-testing.
And importantly, I found beautiful abstractions for HTTP requests and responses. For clients, requests are outgoing data. For servers, responses are also outgoing data. Since response statuses can be expressed as a pseudo :status
header, we can define outgoing data as follows:
data OutObj = OutObj { outObjHeaders :: [Header] -- ^ Accessor for header. , outObjBody :: OutBody -- ^ Accessor for outObj body. , outObjTrailers :: TrailersMaker -- ^ Accessor for trailers maker. } data OutBody = OutBodyNone -- | Streaming body takes a write action and a flush action. | OutBodyStreaming ((Builder -> IO ()) -> IO () -> IO ()) | OutBodyBuilder Builder | OutBodyFile FileSpec
For the client libarary, Request
is just a wrapper data type:
-- | Request from client. newtype Request = Request OutObj deriving (Show)
Response
in the server library is also a wrapper:
-- | Response from server. newtype Response = Response OutObj deriving (Show)
The same discussion can be done for incoming data thanks to pseudo headers including :method
and :path
:
type InpBody = IO ByteString -- | Input object data InpObj = InpObj { inpObjHeaders :: HeaderTable -- ^ Accessor for headers. , inpObjBodySize :: Maybe Int -- ^ Accessor for body length specified in c ontent-length:. , inpObjBody :: InpBody -- ^ Accessor for body. , inpObjTrailers :: IORef (Maybe HeaderTable) -- ^ Accessor for trailers. }
Here comes Response
for the client library:
-- | Response from server. newtype Response = Response InpObj deriving (Show)
And Request
in the server library is:
-- | Request from client. newtype Request = Request InpObj deriving (Show)
I shouted about this experience:
Based on #QUIC experience, now I'm implementing HTTP/2 client library in #Haskell. Beautiful abstraction was found. Based on this activity, I'll tackle HTTP/3 and QPACK.
— 山本和彦 (@kazu_yamamoto) 2020年3月18日
HTTP/3 client and server library
Now it was time to implement HTTP/3. Thanks to Request
and Response
from HTTP/2 library and QUIC library itself, I was able to concentrate on how to manipulate multiple streams. Suddenly, I got an insight about QUIC streams:
I got an insight where a #QUIC stream is like a TCP connection while a QUIC connection is an OS. So, the classical socket API style can be applied for QUIC streams. Haskell QUIC library takes this approach.
— 山本和彦 (@kazu_yamamoto) 2020年4月29日
Now QUIC library in Haskell provides an abstract data type for streams:
data Stream
Clients can creates a Stream
like sockets:
stream :: Connection -> IO Stream unidirectionalStream :: Connection -> IO Stream
A server get a Stream
when a new QUIC connection comes:
acceptStream :: Connection -> IO (Either QUICError Stream)
Data can be received and sent though Stream
:
-- return "" when FIN is received recvStream :: Stream -> Int -> IO ByteString sendStream :: Stream -> ByteString -> IO () -- Sending FIN shutdownStream :: Stream -> IO ()
With these APIs, I was able to develop HTTP/3 really fast. In the sense where a lightweight thread is used per stream, programming HTTP/3 is like HTTP/1.1. In the sense where frames are used, programming HTTP/3 is like HTTP/2. I felt that my long career for HTTP/1.1 and HTTP/2 is converged in HTTP/3!