あどけない話

Internet technologies

Developing network related libraries in Haskell in 2022FY

This article is my annual report of 2022FY(fiscal year in Japan; from April 2022 to March 2023). My mission in IIJ is contribute to standardizations of new network protocols by their implementations. As you may know, I maintain some network-related libraries in Haskell.

HTTP/2

Background: the http2 library originally provided HTTP/2 frame and HPACK encoder/decoder only. This was integrated into Warp to provide the HTTP/2 server functionality. This functionality was extracted into the server side in http2. Then the client side was implemented.

Version 3.0.x were released in 2021FY to fix the vulnerabilities reported in "HTTP/2 implementations do not robustly handle abnormal traffic and resource exhaustion".

Version 4.0.0

RFC 9113 was published in June 2022 and obsoleted RFC 7540. The main purpose of this version is to catch up this new RFC. The major version up was due to a lot of breaking changes.

Version 4.1.0

The server side has been tested well in the real world but the client side is not. Evgeny Poberezkin kindly reported some bugs relating to streaming in the client side. This version should fix these bugs. I thank him a lot.

The major version up again because the internal data type was changed resulting in the build break of http3. A new version of http3 to catch up this change was also released.

QUIC

Background: the quic/http3 library have been developed since 2019 and were released after RFC9000 was published in 2021.

Version 0.0.x adopts the fusion crypto engine for performance reasons. After releasing version 0.0.x, I noticed that it supports the Intel architecture only. I should have quickly worked around but my interest went to QUIC version 2 and the version negotiation. After implementing these new technologies, I had integrated the fusion and cryptonite to let run the quic library on all platforms.

At this moment, I had a dilemma: I cannot release a new version of quic since the numbers of QUIC version 2 and the trasnport parameter for the version negotiation would change. (The value of QUIC version 2 was 0x709a50c4 in drafts and is 0x6b3343cf finally. It's not 0x00000002 at all!)

Meanwhile, I spend time to support Windows. The UDP implementation of Windows is really awkward. See the following blog article for more information.

Based on this experience, I have released the network-udp library which provides best current practice for UDP clients and servers. (This library is also used in the dnsext-do53 library described later.)

Though RFCs of QUIC version 2 and the version negotiation have not been released, the numbers have been fixed. So, I released the quic library v0.1.0 finally in Feb 2023.

DNS

Background: to implement anti-spam technologies such as SPF and DKIM, I have started implementing the dns library purely in Haskell since 2010. Thanks to GHC's concurrency, the stub resolver functionality is highly concurrent. Fortunately this library is used widely but unfortunately two down-sides were turned out:

  1. Resource records are not extensible: resource records are implemented as a sum type. The third party library cannot extend them. The only way to extend them is to send a pull request to the dns library.
  2. Resource records are not friendly to caching: some resource records use ByteString internally. So, if they are cached for a long time, fragmentation happens.

dnsext

It appeared impossible to maintain backward compatibilities to the dns library. So, new libraries whose prefix is dnsext- were created.

  • dnsext-types: basic types with encoders/decoders. To solve 1), I introduced typeclasses. To fix 2), ShortByteString is used as an internal representation.

The following is the definition of extensible resource records:

class (Typeable a, Eq a, Show a) => ResourceData a where
    resourceDataType :: a -> TYPE
    putResourceData  :: CanonicalFlag -> a -> SPut ()

-- | A type to uniform 'ResourceData' 'a'.
data RData = forall a . (Typeable a, Eq a, Show a, ResourceData a) => RData a

A basic type Domain is defined using ShortByteString instead of ByteString:

data Domain = Domain {
    -- The representation format. Case-sensitive, escaped.
    representation  :: ShortByteString
    -- Labels in wire format. Case-sensitive, not escaped.
  , wireLabels      :: [ShortByteString]
  -- | Eq and Ord key for Canonical DNS Name Order.
  --   Lower cases, not escaped.
  --   https://datatracker.ietf.org/doc/html/rfc4034#section-6.1
  , canonicalLabels :: ~[ShortByteString]
  }
  • dnsext-do53: DNS over UDP port 53

The following is a typical usage of the stub resolver:

> withLookupConf defaultLookupConf $ \env -> lookupA env "www.iij.ad.jp"
Right [202.232.2.180]

My current interest is SVCB(Service Binding)/HTTPS resource records and DNS over new transport protocols.

  • dnsext-svcb: SVCB related resource records. This is an extension example of dnsext-types.
  • dnsext-dox: DNS over HTTP2, HTTP3, TLS and QUIC

After Kei Hibino joined to IIJ, he has focused implementing DNSSEC verifier and a full resolver.

  • dnsext-dnssec: DNSSEC verifier
  • dnsext-full-resolver: a full resolver (aka a cache server)

He wrote three articles on this topic in Japanese (article 1, article 2, article 3).

dnsext-full-resolver provides a command line interface called dug. It shows how iterative resolve works if the -i option is specified:

% dug -i www.iij.ad.jp
resolve-just: dc=0, ("www.iij.ad.jp.",A)
    "a.root-servers.net." ["198.41.0.4","2001:503:ba3e::2:30"]
    "b.root-servers.net." ["199.9.14.201","2001:500:200::b"]
    "c.root-servers.net." ["192.33.4.12","2001:500:2::c"]
        ...
iterative: selected addrs: (198.41.0.4,"jp.",A)
iterative: selected addrs: (2001:503:ba3e::2:30,"jp.",A)
iterative: selected addrs: (199.9.14.201,"jp.",A)
...
    "a.dns.jp." ["203.119.1.1","2001:dc4::1"]
    "b.dns.jp." ["202.12.30.131","2001:dc2::1"]
    "c.dns.jp." ["156.154.100.5","2001:502:ad09::5"]
        ...
iterative: selected addrs: (203.119.1.1,"ad.jp.",A)
iterative: selected addrs: (2001:dc4::1,"ad.jp.",A)
iterative: selected addrs: (202.12.30.131,"ad.jp.",A)
...
    "a.dns.jp." ["203.119.1.1","2001:dc4::1"]
    "b.dns.jp." ["202.12.30.131","2001:dc2::1"]
    "c.dns.jp." ["156.154.100.5","2001:502:ad09::5"]
        ...
iterative: selected addrs: (203.119.1.1,"iij.ad.jp.",A)
iterative: selected addrs: (2001:dc4::1,"iij.ad.jp.",A)
iterative: selected addrs: (202.12.30.131,"iij.ad.jp.",A)
...
    "dns0.iij.ad.jp." ["210.130.0.5","2001:240::105"]
    "dns1.iij.ad.jp." ["210.130.1.5","2001:240::115"]
iterative: selected addrs: (210.130.0.5,"www.iij.ad.jp.",A)
iterative: selected addrs: (2001:240::105,"www.iij.ad.jp.",A)
iterative: selected addrs: (210.130.1.5,"www.iij.ad.jp.",A)
...
    "dns0.iij.ad.jp." ["210.130.0.5","2001:240::105"]
    "dns1.iij.ad.jp." ["210.130.1.5","2001:240::115"]
resolve-just: selected addrs: (210.130.0.5,"www.iij.ad.jp.",A)
resolve-just: selected addrs: (2001:240::105,"www.iij.ad.jp.",A)
resolve-just: selected addrs: (210.130.1.5,"www.iij.ad.jp.",A)
resolve-just: selected addrs: (2001:240::115,"www.iij.ad.jp.",A)
--------------------
;; HEADER SECTION:
;Standard query, NoError, id: 8338
;Flags: Authoritative Answer


;; OPTIONAL PSEUDO SECTION:
;UDP: 1232, Data:[]

;; QUESTION SECTION:
;www.iij.ad.jp.     IN  A

;; ANSWER SECTION:
www.iij.ad.jp.  300(5 mins) IN  A   202.232.2.180

If the -d auto option is specified, dug first resolves SVCB RR and selects DNS over X according to its response.

% dug @94.140.14.140 -d auto www.iij.ad.jp
;; 2a10:50c0::2:ff#443/HTTP/3, Tx:42bytes, Rx:58bytes, 132usec

;; HEADER SECTION:
;Standard query, NoError, id: 43730
;Flags: Recursion Desired, Recursion Available

;; OPTIONAL PSEUDO SECTION:
;UDP: 0, Data:[]

;; QUESTION SECTION:
;www.iij.ad.jp.     IN  A

;; ANSWER SECTION:
www.iij.ad.jp.  300(5 mins) IN  A   202.232.2.180

We don't have a releasing plan at this moment. We should concentrate the field test of the full resolver in 2023FY.