あどけない話

Internet technologies

Myth and truth in Haskell asynchronous exceptions

This article explains my best current practice on asynchronous exceptions in Haskell using the standard library - Control.Exception. Other libraries for safe exceptions are out of scope. The followings are the definitions of two kinds of exceptions.

  • Synchronous exceptions are ones raised by your actions. You can throw a synchronous exception to yourself via throwIO.
  • Asynchronous exception are ones thrown by other threads. You can re-throw them to yourself via throwIO when you catch them.

Before talking about asynchronous exceptions, let's start with synchronous exceptions.

Synchronous exceptions

You can catch synchronous exceptions by catch, handle and try:

catch :: Exception e => IO a -> (e -> IO a) -> IO a
handle :: Exception e => (e -> IO a) -> IO a -> IO a
try :: Exception e => IO a -> IO (Either e a)

As you can see from this signature, the handler can take only one type. Here is an example:

import Control.Exception
import GHC.IO.Exception
import System.IO.Error

handled_action :: IO ()
handled_action = your_action `catch` handler
  where
    handler :: IOException -> IO ()
    handler e
        -- Bad file descriptor
        | ioeGetErrorType e == InvalidArgument = return ()
        -- Connection refused
        | ioeGetErrorType e == NoSuchThing = return ()
        -- Print it for debugging
        | otherwise = print e

You can define your own exception as follows:

import Control.Exception

data EarlyReturn = EarlyReturn deriving (Eq, Show)
instance Exception EarlyReturn

The important methods of the Exception class are as follows:

class (Typeable e, Show e) => Exception e where
    toException   :: e -> SomeException
    fromException :: SomeException -> Maybe e

instance Exception EarlyReturn derives the methods properly.

The following code implements early return like other languages:

import Control.Monad (when)

early_return :: IO ()
early_return = handle ignore $ do
    when condition1 $ throwIO EarlyReturn
    when concition2 $ throwIO EarlyReturn

    action1
    action2
    action3
    ...
  where
    ignore EarlyReturn = return ()

If you want to catch two or more types, you can use catches:

catches :: IO a -> [Handler a] -> IO a 

In my opinion, catches is a little bit hard to use. Rather, I usually use PatternGuard. The following example defines the Break exception as well as EarlyReturn:

{-# LANGUAGE PatternGuards #-}

data Break = Break deriving (Eq, Show)
instance Exception Break

handled_action2 :: IO ()
handled_action2 = your_action `catch` handler
  where
    handler :: SomeException -> IO ()
    handler se
        | Just Break <- fromException se = return ()
        | Just EarlyReturn <- fromException se = return ()
        -- Print it for debugging
        | otherwise = print se

The super data type, SomeException is defined as follows:

{-# LANGUAGE ExistentialQuantification #-}

data SomeException = forall e . Exception e => SomeException e

The constructor SomeException is a kind of container which can contain several types and provides a single type, SomeException. The following example converts IOError and ErrorCall to SomeException:

ghci> import Control.Exception

ghci> :type userError "foo"
userError "foo" :: IOError
ghci> :type SomeException (userError "foo")
SomeException (userError "foo") :: SomeException

ghci> :type ErrorCall "bar"
ErrorCall "bar" :: ErrorCall
ghci> :type SomeException (ErrorCall "bar")
SomeException (ErrorCall "bar") :: SomeException

fromException returns Just if SomeException contains an expected exception type. Otherwise, it returns Nothing:

ghci> let se = SomeException $ ErrorCall "bar"
ghci> fromException se :: Maybe ErrorCall
Just bar
ghci> fromException se :: Maybe IOError 
Nothing

A downside of SomeException is that asynchronous exceptions are caught as well as synchronous exceptions. This could introduce nasty bugs. We will resolve this issue according to rule 3 later.

Asynchronous exceptions

If your code acquires a resource, it must be released after your job. The following is an example of unsafe code to obtain a resource:

unsafe_action_with_resource = do
   x <- acquire_resource
   use x
   release_resource x

If use receives an asynchronous exception, x is not released, thus leaked. To prevent this resource leak, you should use bracket:

bracket :: IO a -> (a -> IO b) -> (a -> IO c) -> IO c

The example can be convert into asynchronous-exception-safe code with bracket:

safe_action_with_resource = bracket acquire_resource release_resource use

Some people (including me) misunderstand/misunderstood that asynchronous exceptions are not delivered into the resource release clause. But this is not quit right.

Consider the following situation:

  • You run a program in Haskell in a terminal
  • Your program is blocked in the resource release clause of bracket.
  • You type Control-C to stop the program on the terminal.

Control-C is an asynchronous exception (in particular UserInterrupt :: AsyncException). If this asynchronous exception is not delivered to the resource release clause, you cannot stop your program.

But Control.Exception is well-defined. Blocking actions such as takeMVar is interruptible by asynchronous exceptions. In other words, non blocking code cannot be interrupted by asynchronous exceptions in the clause. As you will see later, timeout is defined with an asynchronous exception. You can use even timeout in the resource release clause. This is a crucial point to implement graceful-closing of network connections as it is used in the resource release clause typically.

As you understood, the semantics of Control.Exception is clear and well-defined. That's why this article is stick to it.

As described above, if you catch asynchronous exceptions which others define via SomeException, nasty bugs happens. For instance, the async package uses an asynchronous exception, AsyncCanceled. If you run two threads which ignore AsyncCanceled via concurrently, these threads would be leaked.

So, you should obey the following three rules:

Rule 1

If you define an asynchronous exception, you must explicitly define the methods of the Exception class:

import Control.Exception

data Timeout = Timeout deriving (Eq, Show)

instance Exception Timeout where
    toException = asyncExceptionToException
    fromException = asyncExceptionFromException

If we follow this rule, asynchronous exceptions can be distinguished from synchronous exceptions with the following function:

isAsyncException :: Exception e => e -> Bool
isAsyncException e =
    case fromException (toException e) of
        Just (SomeAsyncException _) -> True
        Nothing -> False

Rule2: catch the asynchronous exception which you are using

If you don't catch, the exceptions are leaked from your threads. GHC RTS catches them and displays them to stdout.

The following is an example to define naive timeout:

import Control.Concurrent
import Control.Exception

data Timeout = Timeout deriving (Eq, Show)

instance Exception Timeout where
    toException = asyncExceptionToException
    fromException = asyncExceptionFromException

timeout :: Int -> IO a -> IO (Maybe a)
timeout t action = do
    pid <- myThreadId
    handle handler $ bracket (spawn pid) kill $ \_ -> Just <$> action
  where
    spawn pid = forkIO $ do
        threadDelay t
        throwTo pid Timeout
    kill tid = throwTo tid ThreadKilled
    handler Timeout = return Nothing

In this example, an asynchronous exception, ThreadKilled, is defined and caught by handler.

Rule 3: don't catch asynchronous exceptions of others

Exceptions of others have their own semantics. If you catch them, your code does not work well.

If you use SomeException for catch, handle or try, check if the caught exception is asynchronous. And re-throw it via throwIO if asynchronous. You can use the following pattern for this purpose:

{-# LANGUAGE PatternGuards #-}

handled_action3 :: IO ()
handled_action3 = your_action `catch` handler
  where
    handler :: SomeException -> IO ()
    handler se
        | isAsyncException se = throwIO se -- HERE HERE HERE
        | Just Break <- fromException se = return ()
        | Just EarlyReturn <- fromException se = return ()
        -- Print it for debugging
        | otherwise = print se

Event-poll programming

Recently, Kei Hibino, my colleage, spoke eloquently to me about the dangers of asynchronous exceptions. Asynchronous exceptions are thrown to us even when we are not expected them. Rather, he suggested event-poll programming.

Let's consider the recv function of the network library with timeout. This is an essential part of the graceful-closing function (read Implementing graceful-close in Haskell network library in detail):

import Data.ByteString
import Network.Socket
import Network.Socket.ByteString
import System.Timeout

timeoutRecv :: Socket -> Int -> IO (Maybe ByteString)
timeoutRecv sock usec = timeout usec $ recv sock 1024

As I explained, timeout uses an asynchronous exception. How can we implement this function without asynchronous exceptions?

The idea is as follows:

  • Prepare one STM action and ask the system TimerManager to wake up me via the action on timeout.
  • Prepare another STM action and ask the system IOManager to wake up me via the action when the socket is ready for read.
  • Race two managers and check if which one is a winner via the composition of these STM actions. If TimerManager wins, return Nothing from IO. Otherwise, the winner is IOManager. This means that data is available for the socket. So, call recv.

The following code implements this idea. I don't explain this code in detail but please read carefully if you are interested.

import Control.Concurrent.STM (
    atomically,
    newEmptyTMVarIO,
    orElse,
    putTMVar,
    takeTMVar,
 )
import Control.Exception (bracket)
import Data.ByteString (ByteString)
import GHC.Event (getSystemTimerManager, registerTimeout, unregisterTimeout)
import Network.Socket (Socket, waitAndCancelReadSocketSTM)
import Network.Socket.ByteString (recv)

data Wait = MoreData | TimeoutTripped

recvEOFevent :: Socket -> Int -> IO (Maybe ByteString)
recvEOFevent sock usec = do
    tmmgr <- getSystemTimerManager
    tmvar <- newEmptyTMVarIO
    bracket (setupTimeout tmmgr tmvar) (cancelTimeout tmmgr) $ \_ -> do
        bracket (setupRead sock) cancelRead $ \(rxWait', _) -> do
            let toWait = do
                    takeTMVar tmvar
                    return TimeoutTripped
                rxWait = do
                    rxWait'
                    return MoreData
            waitRes <- atomically (toWait `orElse` rxWait)
            case waitRes of
                TimeoutTripped -> return Nothing
                MoreData -> Just <$> recv sock 1024
  where
    setupTimeout tmmgr tmvar =
        registerTimeout tmmgr usec $ atomically $ putTMVar tmvar ()
    cancelTimeout = unregisterTimeout
    setupRead = waitAndCancelReadSocketSTM
    cancelRead (_, cancel) = cancel

I'm trying to get rid of asynchronous exceptions as much as possible from my network libraries by introducing the event-poll programming.

Labeling threads in Haskell

GHC 9.6 provides a function to list up the current threads finally. The function is listThreads exported from the GHC.Conc.Sync module. listThreads is a killer debug method for thread leaks.

If you have Haskell programs which run for a long time, it's quite nice to provide feature to monitor threads with the following functions:

import Data.List (sort)
import Data.Maybe (fromMaybe)
import GHC.Conc.Sync (ThreadStatus, listThreads, threadLabel, threadStatus)

printThreads :: IO ()
printThreads = threadSummary >>= mapM_ (putStrLn . showT)
  where
    showT (i, l, s) = i ++ " " ++ l ++ ": " ++ show s

threadSummary :: IO [(String, String, ThreadStatus)]
threadSummary = (sort <$> listThreads) >>= mapM summary
  where
    summary t = do
        let idstr = drop 9 $ show t
        l <- fromMaybe "(no name)" <$> threadLabel t
        s <- threadStatus t
        return (idstr, l, s)

The following is an example of how printThreads displays a list of thread status:

1 (no name): ThreadFinished
2 IOManager on cap 0: ThreadRunning
3 TimerManager: ThreadBlocked BlockedOnForeignCall
4 main: ThreadRunning
5 accepting: ThreadBlocked BlockedOnMVar
6 server:recv: ThreadBlocked BlockedOnForeignCall
7 server:gracefulClose: ThreadRunning

Let's label threads

Threads spawned via forkIO or others do not have its label by default. Threads without label displayed "(no name)" in the example above. If there are a lot of threads without label, debugging is hard. So, I have already asked GHC developers to label threads created in the libraries shipped with GHC.

I would also like to ask all library maintainers to label threads if forked. You can use the following code to label your threads:

import Control.Concurrent (myThreadId)
import GHC.Conc.Sync (labelThread)

labelMe :: String -> IO ()
labelMe lbl = do
    tid <- myThreadId
    labelThread tid lbl

labelThread is a very old function. So, you can use it without worrying about GHC versions.

labelThread override the current label if exists. If you don't want to override it, use the following function:

{-# LANGUAGE CPP #-}

import Control.Concurrent (myThreadId)
import GHC.Conc.Sync (labelThread, threadLabel)

labelMe :: String -> IO ()
#if MIN_VERSION_base(4,18,0)
labelMe name = do
    tid <- myThreadId
    mlabel <- threadLabel tid
    case mlabel of
        Nothing -> labelThread tid name
        Just _ -> return ()
#else
labelMe name = do
    tid <- myThreadId
    labelThread tid name
#endif

Unfortunately, the first appear of threadLabel is GHC 9.6. So, #if is necessary.

ThreadFinished

Threads in the ThreadFinished status should be GCed quickly. If you see a long-lived threads in this status, their ThreadIds are held somewhere. Surprisingly, ThreadId is not integer but reference! The following is an example that a WAI timeout manger holds ThreadIds, resulting in thread leaks.

10150 WAI timeout manager (Reaper): ThreadBlocked BlockedOnMVar
10190 Warp HTTP/1.1 192.0.2.1:43390: ThreadFinished
10191 Warp HTTP/1.1 192.0.2.1:43392: ThreadFinished
10193 Warp HTTP/1.1 192.0.2.1:43404: ThreadFinished
10202 Warp HTTP/1.1 192.0.2.1:43406: ThreadFinished
10204 Warp HTTP/1.1 192.0.2.1:41256: ThreadFinished

To prevent this thread leaks, hold Weak ThreadId instead. This can be created via mkWeakThreadId provided by Control.Concurrent. To convert Weak ThreadId to ThreadId, use deDefWeak exported from GHC.Weak.

More labels

The ThreadBlocked constructor of the ThreadStatus type contains BlockReason. It has the following constructors:

  • BlockedOnMVar
  • BlockedOnBlackHole
  • BlockedOnException
  • BlockedOnSTM
  • BlockedOnForeignCall
  • BlockedOnOther

It's nice if we can label MVar via labelMVar and BlockedOnMVar contains its label. STM data types should follow this way, too.

A new architecture for QUIC in Haskell

In typical UDP programming, unconnected sockets are used both in the client and server sides. sendto() is used to specify a peer address while recvfrom() is utilized to receive a peer address.

In the quic library in Haskell, I used connected sockets for performance reasons. If a connected socket is created for a QUIC connection, recv() can receive data from a specific peer. This means that data is dispatched in the kernel. To understand UDP connection in detail, please read Accepting UDP connections.

As I described in Implementing QUIC in Haskell, I found a drawback of this approach. When a client migrates networks, for instance, from a cell phone network to WiFi, the quic library has to detect the new network interface gets available. It is hard to implement a cross-platform scheme for this detection. So, I added another mode to use a unconnected socket and sendto() for clients.

One of my colleagues told me a drawback of servers recently. The quic library assumes that NAT rebindings do not occur during a connection creation. Once a QUIC connection is created, a server can handle NAT rebidings. What he found is there are NAT boxes which change ports very quickly.

I had to admit that the connected socket approach is not feasible. Therefore, all code for connected sockets were removed and a new approach with unconnected sockets is introduced. Note that sendmsg() and recvmsg() are used instead of sendto() and recvfrom() to work with load balancers of DSR (Direct Server Return).

The quic library version 0.2.0 or later provide this new architecture.

A new architecture for HTTP/2 in Haskell

GHC 9.6 provides listThreads finally. Just for curious, I have implemented a thread monitor in http2-server, a test command tool for the http2 library in Haskell. This revealed that huge numbers of Haskell lightweight threads are used. See the following picture of servers from Experience Report: Developing High Performance HTTP/2 Server in Haskell.

The old architecture makes use of the worker pool. Workers of a fixed number are spawned by the worker manager in advance. A worker takes an HTTP request from the input queue, works for the request, generates an HTTP response and then enqueues the response to the output queue.

Then sender dequeues an HTTP response, fills the output buffer with the available data of the response, flushes the output buffer if necessary, and enqueue the response to the output queue again if it has more data. If the flow control window in the stream level is closed for the HTTP/2 stream, it spawns a waiter thread. The waiter thread wait until the window gets open then enqueues the response again.

For the response of the streaming type, the sender also checks if the streaming data is available. If not, the sender spawns a waiter. The thread monitor revealed that the number of waiters is much larger than I expected. This ruins the saving number of worker threads.

To avoid the thread number explosion, I need to give up the worker pool. Instead, a worker is spawn for an HTTP/2 stream on demand. The sender pushes a response back to the corresponding worker. The worker itself takes care of the flow control window in the stream level or the availability of streaming data.

This new architecture simplified the code drastically:

  • The worker manager is not necessary anymore.
  • The code to go to the next request safely is removed from the worker.
  • The sender should only take care of the flow control window in the connection level.

Version 5.3.0 or later of the http2 library provides this new architecture.

Changes to the paper in Haskell Symposium 2016

  • Client and server code was extracted from Warp to the http2 library. See "HTTP/2 server library in Haskell" and "Implementing HTTP/3 in Haskell" for more information.
  • Section 3 "Priority" is outdated because the priority feature was removed from the HTTP/2 specification. To avoid vulnerability, the priority code was eliminated from the http library.
  • Section 4 "HTTP/2 Implementation in Warp" is also outdated as described in this particle. In particular, Section 4.1 "Optimistic Enqueueing" is meaningless since the number of workers is controlled by SETTINGS_MAX_CONCURRENT_STREAMS (maxConcurrentStreams, default: 64).
  • In Figure 7 and Figure 8 in Section 5 "Evaluation", the stack of Haskell threads consumes large memory. This is because the default value of the stack chunks in GHC (32KiB) is not kind. With this settings, the stack grows like 1KiB, 33KiB, 65KiB, etc. 33KiB is too large as the next step for server usage. Recently, I specify -kc2k to my project code where the step is 1KiB, 3KiB, 5KiB, etc.

I'm a little bit disappointed since I should admit that it's hard for me to write a solid paper about software.

Acknowledgment

I thank Edsko de Vries and Finley McIlwaine for testing the new architecture. They are big contributors to the http2 library in Haskell.

山口でのエギング振り返り

「分かってしまうと、なぜ分からなかったのか分からなくなる」

と言うわけで、いつもは技術的なことしか書かないけれど、今回は「なぜ以前はアオリイカが釣れなかったのか」を忘れない内に書き留めておく。

2022年

親の介護のため、2022年の3月末に、東京から故郷の山口へ引っ越した。瀬戸内の街である故郷では子供達と釣りを楽しもうと思い、「生餌の付けられない子供と一緒にやるにはエギングがいいかな」という軽い気持ちで始めた。

湖畔のキャンプのときに買った2,000円ぐらいのバスロッドを使い、格安のエギを買って5月ごろにエギングをやってみたが、まったく釣れなかった。今思えば当たり前だけど。釣れないので子供達は飽きてしまい、エギングには付き合ってくれなくなった。

秋になり、娘の習い事を待つ間に、近くの漁港に行って、20分だけエギングをやった。しかし、何度やっても釣れない。言い訳できないように、すごく釣れると言うエギ王K ムラムラチェリー2.5号を買い、「今日釣れなかったらやめる」と言う気持ちで、明かりの消えた漁港で竿を振った。10月6日の夜のことだ。

「これがラスト」と思って投げたエギを、ただ巻きしていると急に重くなった。初めはゴミがかかったのか思ったが、何か違う。足元でヘッドライトを当ててみるとアオリイカで、無造作に引き上げた。初めて釣れたというのに、当たりが取れなかったせいか、あまり嬉しくはなかった。

よくやめなかったと思う。このアオリは、エギングの神様が「エギングをやめるな」と与えた1杯だったのかもしれない。

2023年

介護や子供の世話が忙しく、あまり記憶がないのだけれど、春はエギングをやならなかったと思う。このころ、「ウチの近くにはアオリイカなんて、ほとんどいないんだ」と思い込んでいた。釣れると聞いたエギ王K 軍艦グリーンも、まったく市場に出回ってなかった。

9月24日の親戚の集まりで、Tさんから「俺は港の藻がある方側で2杯釣ったぜ」と言われ、少しやる気が出て、また待ち時間の間に行ってみた。下手過ぎて、すぐにエギを根掛かりさせてしまうので、ジュンテンドーで買った安いエギを単にただ巻きしていた。これまた、「これで帰る」と思って投げた一投で、生命感のあるピリピリとした当たりがあり、人生2杯目のアオリイカが釣れた。驚くべきことに、写真を撮っておらず、何日のことだったかはもう分からない。

この秋3杯釣ったら自分へのご褒美で、エギングロッドとリールを買うと決めていたが、釣具のポイントの店長さんと話していると、今すぐ欲しくなって、エントリーモデルを買ってしまった。竿は色々調べて決めていた「シマノ セフィア BB S83ML」、リールは店長の合わせた方がいいと言うススメに従い「シマノ セフィア BB C3000S DH」にした。

この後、フリーリグでチヌを釣ることに時間を使ってしまったので、2023年も釣れたのは秋イカ1杯だけだった。

2024年5月

4月に父を送り出し、雑多な手続きもこなして落ち着き始めた頃、またTさんから「今年のアオリはデカいらしい。俺は釣ってないけど」と言われ、夜にポイントを回ってみた。あるポイントには驚きの光景が広がっていた。たくさんの車が停まっており、椅子が丸く並べられていて、キャンプファイヤー状態。地面には、これまで見たこともない大きなイカが、「釣れるのが当たり前」みたいな感じで、あちらこちらに無造作に置かれていた。このとき初めて、自分の故郷に春イカの一級ポイントがあることを知った。

イカは自分の近くにいるし、竿とリールには投資したし、エギ王K 軍艦グリーンの在庫も復活した。これで釣れないのは、自分の腕が悪からだと素直に信じれるようになり、余裕が出てきた時間を使って、足繁くポイントに通った。そしてとうとう5月29日、春の大きなイカを釣り上げた。モンゴウだけど。

引き上げても横抱きしたエギを離さず、エギに噛み跡を残してくれた君には感謝しかない。モンゴウは細く切らないと美味しくないことを知らずに、息子には「美味しくない」と言われてしまった。まずかったのは、僕の調理の腕だ。許してくれ。

僕にも大きなイカが釣れることが分かり、火がついた。どうしても春のアオリイカが釣りたい。上手な人たちに混じって初心者の自分が釣るには、違うことをするしかないと思い、youtubeで見たスローエギング(リフト&フォールみたいな感じ)をやっみることにした。

次の日の夜、よい場所に入れたので、そこだけ波が立っている場所に軍艦グリーン 3.5号 シャローを投げ、ゆっくりゆっくり誘っていると、もわーん、もわーんとした生命反応を感じた。どう合わせていいのか分からず、乗ったのかどうかも分からないので、我慢してゆっくりゆっくりエギを動かし、「もう逃げたかな」と思って巻き始めたら、急に重くなり一回だけ走った。

水面に現れた姿を周りの人が見付けて「アオリだ!」と言う。かなり近づいたとき、ようやく僕にも見えた。「触腕一本だから気を付けて」と言われるがどうしていいのか分からない。周りの上手い方に、「横に滑らすように」と教えてもらい、何度もやり直して、ようやくタモ入れできる体制に持ち込んで、タモ入れしてもらった。

苦節2年を経た2024年5月30日、ついに春のアオリイカ(オス 926g)を釣り上げた。本当に嬉しかった。

2024年6月

エギング歴の長い方とたまたまポイントで2人きりになったときのこと。僕のしゃくり方がおかしいと指摘していただたいた。「ショートジャークをしているつもりだろうけど、ハンドルを回しているから、エギはまっすぐにしか動かず、すぐに戻って来てしまう。竿を回すようにすると、エギは斜め前方に動くから、岸に向かっての移動距離は短くなり、イカを多く誘える」と教えていただいた。ショートジャークはエギをダートさせるため、つまり横に動かすのが目的だとまったく分かっていなかった。

6月も半ばを過ぎると、多くの人がまったく釣れなくなり、釣り人の数も少なくなった。僕は大潮にはまだアオリが入ってくるだろうと信じ、気になっていたパタパタを買って、6月19日の夕刻に海へ向かった。すると、奇跡的に一投目で釣れた。安定のしゃくり合わせ。両隣の人も、僕の真似をしてパタパタに変えると釣れて3連チャン。時合だったのだ。1172gと夢のキロアップ! メスを持ち帰ってはいけないというルールはないそうで、心に余裕のない僕は持って帰ることにした。(クワガタのメスは取らないよ。本当だよ!)

僕が行くポイントには、みんなが釣れてない中で必ず釣って帰る人がいる。その人に話しかけてみると、「エギング歴25年で、ほとんどボウズはない」と言う。その人の隣で竿を振りながらいろいろ教えてもらった。「君のエギはダートはしているが、跳ね上がっていない」と言う。「2段しゃくりと言う言葉は古いのかもしれないけれど、上に動かさないとイカのスイッチが入らない」そうだ。「潮が動いているかはゴミの位置を観察したら分かる」。「エギはプロスペックの薩摩オレンジで、ローテーションはほとんどしない」。「フケ当たりっていうのは、こんな感じ」と実演してくださった。

そのときから、エギを動かすときに、跳ね上げとダート、つまり縦の動きと横の動きを意識するようになった。すると、その日の内に1375gの大物が釣れた。モンゴウだけどね。6月29日、エギはエギ王K 金アジ 3.0号。

2024年7月

7月4日にまた同じエギでモンゴウが釣れた。この日は自分しかいなかったので、初めて一人でタモ入れしたけど、イメージトレーニングはばっちりで、手際よく引き上げた。計量を忘れしまったので、重さは分からないが、中ぐらい。

そして冒頭の呟きをしたわけだ。次の日にしゃくり合わせでかかったイカは、走る走る。ドラグが鳴り止まず、かなり期待したのだけれど、浮き上がって来たのは873gのモンゴウだった。エギは、通販で手に入れたプロスペック 薩摩オレンジ 3.5号!

どうしてもアオリの3杯目を釣りたい。僕に残された手は朝マズメのみ。夜の家事を早めに済ませ22時には就寝して、7月8日は4時に起きて海に向かう。最初はラインが見えない程暗かったが、明るくなってきた。長雨のせいで潮は濁っているから、底は澄んでいると願って薩摩オレンジで底を攻める。すると、ラインが一瞬フケるのが見え、間髪入れずに合わせると乗った!396gと小さいが、目標の3杯達成!

同日、高校の同級生から「今日は呑み会の日だ」と連絡があり、「前回日程を決めてなかったんだから知らんがな」と思いつつ、子供の送迎もないので、アオリとモンゴウのサクを店に持って行った。僕はアオリを自慢しに来ているのに、みな口を揃えて「モンゴウがうまい」と言う。同席していた親戚のRさんから「モンゴウをもう1杯釣れ」と指令が下った。あの方向に金アジを投げて底を攻めれば釣れると分かっていたので、「釣りますよ」と安請け合いをする。

寝不足なので次の日は普通の時間まで寝ると決めていたのに、4時に目が覚め雨は降っていない。エギングの神様に「海に行け」と言われた気がしたので、昨日と同じようにポイントに向かう。ノルマのモンゴウを釣るために、金アジを投げてしゃくるが、ラインが見えずしっくりこないので、回収を始めた。するとエギの後をロケットのように追ってくるイカがいて、ヘッドライトを当てたのに逃げない。慌ててエギを止めてフォールさせると抱いた!1296gのモンゴウに人生初のサイト・エギングをきめてノルマを達成。

墨跡を掃除しているとラインが見える程度に明るくなってきた。時合だと感じ、エギを薩摩オレンジに変えて底を攻める。風があるのでやりずらいが、ラインが一瞬だけフケたのを見逃さなかった。1089gとキロアップのアオリ! 1日2杯釣れるなんて夢のようだ。

釣り人も少なくなって快適だし、(ここがではなく、一般論では)夏もアオリが釣れないことはないらしいので、7月いっぱいはエギングを続けて、8月は休み、9月から秋イカを狙おうと考えている。

なぜ釣れなかったのか

まぁ結局、エギングに対する態度も中途半端だったし、何も分かっていなかった。

  • エギングにかける時間が短過ぎた → 時間をかけるしかありません。教えてくれる人に出会えると素敵ですね
  • ロッドやリールにお金をケチった → 3万円ぐらい出して、エントリーモデルを買いましょう。安い竿でエギングをやるのは時間の無駄です
  • ロストばかりするので、高いエギを買わなかった → 鉄板の「エギ王K」とか「パタパタ」とかを買いましょう。道具の中でエギが一番大切です
  • ロストを回避する方法と根掛かりしたときに回収する方法が分かってなかった → 底に着いたときの最初は強くしゃくるのをやめましょう。藻にかかったら、竿を斜め上に片手で細かくしゃくり続けると外れます
  • 自分の近くにはアオリイカはあんまりいないと間違った認識をしていた → 瀬戸内にもアオリイカはいます。日本海側に行く必要はありません
  • 縦の動きである「跳ね上げ」と横の動きである「ダート」の違いも分かっていなかったし、認識していなかったから動かし方も知らなかった → 「ワンピッチ・ショート・ジャーク」と「2段しゃくり」を覚えましょう
  • 潮が動くという意味も分かってなかった → ゴミを観察して短時間で移動したら流れてます。エギにかかる抵抗で感じることもできます
  • どの潮のどの潮位のときに釣れるか分かっていなかった → 生意気ですが、企業秘密とします
  • 当たりの取り方が分かっていなかった → 僕も夜だとよく分からないので、ラインが見える時間帯にやって、フケ当たりを見逃さないようにしましょう。底に着いたのか、フケ当たりなのか分からなくても、とにかくフケたら合わせるのです。夜でも運がよければ、生命感のある当たりも来るかもれません

最後に、あるロッククライマーが言った言葉で、このブログを締めくくることにする。

「長く続けていれば、いいことあるよ」

Happy Eging!

Status report of dnsext

This article reports the current status of the dnsext packages in Haskell. If you don't know what dnsext is, please read "Developing network related libraries in Haskell in 2022FY" first. The purpose of this project is provide DNS full resolver (cache server).

bowline

Our DNS full resolver is now called bowline named after the king of knots. (I used to climb rocks with double eight knot but I like bowline.) Most importantly, our DNSSEC verifier has been integrated into bowline by Kei Hibino.

New features

To make bowline practical, the following features have been added:

  • Configuration: bowline.conf is in the key-value format. Especially, local-zone:, local-data:, root-hints: and trust-anchor-file: can be specified.
  • Logging: the standard way of logging for DNS servers is DNSTAP whose format is protocol buffer and transport is fast stream. Instead of using other libraries, we implement them in dnsext-utils.
  • Statistic monitoring: Prometheus is supported.
  • Web API: the recent trend for server management is containers. When servers run in containers, the traditional signal scheme is not feasible. So, bowline provides web API for reading statistic, reloading, etc.

DNS transport

To protect privacy, the transport between DNS full resolvers and stub resolvers should be encrypted. The send-receive API of tls and quic is suitable to implement DoT (DNS over TLS) and DoQ (DNS over QUIC). However, the worker model of http2 and http3 is inefficient for DoH (DNS over HTTP). To emulate the send-receive API, runIO is implemented and provided from Internal module of http2. Unfortunately, I have no idea on how to implement runIO for http3 at this moment.

While verifying safety of http2 and quic, I noticed that not all cases of flow control are covered. The following should be implemented for stream numbers in a connection, amount of sending/receiving data in a connection and amount of sending/receiving data in a stream:

  • Telling the limit of receiving data to the peer in proper timing
  • Closing the connection if the receiving data reaches the limit
  • Sending data with the respect of the limit of sending data

To extract common patterns of flow-control, the network-control package is created. With network-control, http2 and quic have covered the all cases.

Refactoring and testing

The code for iterative queries was huge and monolithic. So, it was divided into multiple modules with the help of calligraphy which can visualize call-graph of functions.

dnsperf is used to measure server performance and to run stress testing. We noticed that stacks of Haskell lightweight threads consume huge memory. Their initial size of 1 KiB. When the limit are reached, they glow 33 KiB since the next chunk size is default to 32 KiB. In my opinion, this value is too big because threads might use only 2 KiB, for instance. So, we specify -kc2k (2 KiB) as an RTS option so that the size of stack glows 1KiB, 3 KiB, 5 KiB, 7 KiB and so on.

dug

dug is a command line interface for DNS queries. Of course, it can resolve records for a target domain using UDP as a stub resolver:

% dug www.iij.ad.jp aaaa
;; 2001:a7ff:5f01:1::a#53/UDP, Tx:42bytes, Rx:196bytes, 34usec
...
;; ANSWER SECTION:
www.iij.ad.jp.  300(5 mins) IN  AAAA    2001:240:bb81::10:180

The characteristics of dug are as follows:

  • Queries can be sent with DoT, DoQ and DoH if the -d option is specified.
  • Such a transport is automatically selected by parsing SVCB RRs if the -d auto is specified.
  • It can execute the iterative query algorithm used in bowline if the -i option is specified.

The followings are the new feature added in 2023FY:

  • tcp is added in addition to auto, doq, dot etc for the -d option.
  • The result of DNSSEC is displayed with colors if the --demo option is specified.
  • The query result is displayed in the JSON format if the --json option is specified.

Releasing tls library version 2.0.0 in Haskell

I needed to implement the session ticket mechanism for my project. In addition to this coding, I decided to improve the tls library in Haskell drastically. So, I have spent three months to do so and finally released tls vresion 2.0.0. This version is secure by default and its code readability is improved. This article explains what changed.

Removing insecure stuff

tls version 1.9.0 supports TLS 1.0 and TLS 1.1 in addition to TLS 1.2 and TLS 1.3. RFC 8996 deprecates TLS 1.0 and TLS 1.1. So, they are removed from tls.

TLS 1.2 is considered secure if configured correctly while TLS 1.3 is considered secure by design. To ensure secure configuration, the followings are removed according to "Deprecating Obsolete Key Exchange Methods in TLS 1.2":

  • CBC ciphers
  • RC4 and 3DES
  • DSS(digital signature standard)

The current cipher suites for TLS 1.2 are only:

  • TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  • TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  • TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  • TLS_ECDHE_ECDSA_WITH_AES_128_CCM
  • TLS_ECDHE_ECDSA_WITH_AES_256_CCM
  • TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8
  • TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8
  • TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  • TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256

To prevent the triple handshake attack, the extended main secret defined by RFC7627 is necessary. supportedExtendedMasterSec was default to AllowEMS. It is renamed to supportedExtendedMainSecret and its default value is set to RequireEMS.

I believe that your code can be built without any modifications if you don't customize parameters heavily. If your code cannot be built, I'm sorry, but this breaking changes are intentional to tell that you are using insure parameters for TLS 1.2.

Catching up RFC8446bis

TLS 1.3 is defined in RFC8466 and it is being revised in RFC8466bis. Important changes are:

  • The word "master" is renamed to "main".
  • general_error alert is defined.

tls v2.0.0 catches up RFC8466 bis as much as possible.

Improving API

To send early data in TLS 1.3, clientEarlyData should be used in tls version 1.9.0. Fixed string can be passed through this interface but it is not feasible for real usage since applications decide early data dynamically. With tls version 2.0.0, sendData can now send early data if clientUseEarlyData is set to True.

Client's handshake for TLS 1.3 can now receive the alert of client authentication failure.

Client's bye can now receive NewSessionTicket in TLS 1.3.

Refactoring

handshake was monolithic. To follow the handshake diagram of TLS 1.2 and 1.3, its internal is divided. The result code for TLS 1.2 client looks:

handshake cparams ctx groups mparams = do
    ...
    crand <- sendClientHello cparams ctx groups mparams pskinfo
    ...
    (ver, hss, hrr) <- receiveServerHello cparams ctx mparams
    ...
    case ver of
        TLS13 -> ...
        _ -> do
            recvServerFirstFlight12 cparams ctx hss
            sendClientSecondFlight12 cparams ctx
            recvServerSecondFlight12 ctx

The test framework is switched from tasty to hspec. The quality of each test case is improved.

Also, the following modifications are done:

  • The code is now formatted with fourmolu.
  • PatternSynonyms is introduced for extensibility.
  • The Strict and StritData pragma are specified.

Session Manager

tls 1.9.0 has an abstraction for session management called SessionManager:

data SessionManager {
    sessionResume :: SessionID -> IO (Maybe SessionData)
  , sessionResumeOnlyOnce :: SessionID -> IO (Maybe SessionData)
  , sessionEstablish :: SessionID -> SessionData -> IO ()
  , sessionInvalidate :: SessionID -> IO ()
}

Network.TLS.SessionManager in tls-session-manager version 0.0.4 provides SessionManager for in-memory session DB. When implementing the session ticket mechanism, it appeared that this abstraction is not good enough since there are no way to return tickets. So, SessionManager in tls version 2.0.0 is now:

data SessionManager {
    sessionResume :: SessionID -> IO (Maybe SessionData)
  , sessionResumeOnlyOnce :: SessionID -> IO (Maybe SessionData)
  , sessionEstablish :: SessionID -> SessionData -> IO (Myabe Ticket)
  , sessionInvalidate :: SessionID -> IO ()
  , sessionUseTicket :: Bool
}

Network.TLS.SessionTicket is finally implemented in version 0.0.5.

Interoperability test

To test interoperability with other implementation, I use tls-simpleclient and tls-simpleserver in tls-debug. Unfortunately, I don't have the upload permission for tls-debug to Hackage. Also, it's very inconvenient to build them since they are in the separate package. So, I imported them into the util directory of tls and renamed to client and server. To build them, specify the devel flag to cabal or your favorite command.

client and server are tested with OpenSSL and gnutls both for TLS 1.2 and 1.3.