Chronicle Threads

Thread pool

This library provides a high performance thread pool that shares blocking, monitoring, and busy waiting threads. Busy waiting tasks have HIGH, MEDIUM, or DAEMON (low) priority, or TIMER (fixed rate events) status. All tasks run on a single thread without creating garbage.

See net.openhft.chronicle.core.threads.EventLoop and net.openhft.chronicle.threads.EventGroup.

Event loops and event handlers

An event handler is a routine action that should be executed in response to an event. Therefore, event handlers should be invoked frequently to check whether a specific event has happened, and subsequently the event handler can execute the relevant action in response to the event. In order to trigger frequent invocation of event handlers, an event loop is instantiated and event handlers are added to the event loop. Each event loop can serve multiple event handlers. Figure 1 illustrates a diagram of an event loop that serves several event handlers sequentially.

Figure 1. Event loop and event handlers

You can create and remove event loops and event handlers and add event handlers to event loops as described below:

Creating event handler: An event handler can be created by implementing the EventHandler interface and overriding its action() method so that it executes some actions in response to the event that this event handler is responding to. The action() method returns a boolean value. If the action() method returns true, the event loop is biased to serve the same handler again soon. How soon depends on a variety of factors, and the other work the event loop has to do across the other event handlers. Returning "true" when there is no actual work to do may waste cycles servicing a handler which has nothing to do, at the expense of stealing cycles away from other handlers. Conversely, returning "false" when there is work to do, will effectively increase latency as the event loop will take the "false" as a hint that several other handlers can be serviced ahead of this one.

//Create an event handler class that is ran for 30 times then throws
//InvalidEventHandlerException in order to remove the handler from event loop
public final class TestMediumEventHandler implements EventHandler {

        private int actionCnt;

        @Override
        public boolean action() throws InvalidEventHandlerException {
            if (actionCnt >= 30)
                throw InvalidEventHandlerException.reusable();
            actionCnt++;
            return true;
        }
}

//Instantiate event handler eh0
final TestMediumEventHandler eh0 = new TestMediumEventHandler();

Creating event loop: You can create an event loop by calling the constructor of the class VanillaEventLoop. Then by calling the addHandler() method of this event loop you can add event handlers to the event loop.

//Create event loop el
final VanillaEventLoop el = new VanillaEventLoop(null,
                                                 "test-event-loop",
                                                 PauserMode.busy.get(),
                                                 20,
                                                 false,
                                                 "none",
                                                 EnumSet.of(HandlerPriority.MEDIUM));

The arguments of VanillaEventLoop in the order of appearance are:

final EventLoop parent: The parent event loop if this event loop is in an event group, otherwise "null".
final String name: The name of this thread.
final Pauser pauser: The pause strategy.
final long timerIntervalMS: The pause duration.
final boolean daemon: "true" if this is a daemon thread.
final String binding: Set affinity description, "any", "none", "1", "last-1".
final Set<HandlerPriority> priorities): The set of priorities that this event loop accepts for event handlers that are added to this event loop.

Methods start() and stop(), starts and stops event loops respectively. Event handlers are not executed before starting the event loop even if they have already been added to an event loop.

//Start event loop el.
el.start();

Adding an event handler to an event loop: By calling addHandler() method of an event loop you can add an event handler to an event loop.

//Add event handler eh0 to event loop el
el.addHandler(eh0);

Removing an event handler from an eventLoop: When an event handler is not required anymore and should be removed from the event loop, its action() method should throw InvalidEventHandlerException. The InvalidEventHandlerException.reusable() method returns a reusable, pre-created, InvalidEventHandlerException that is unmodifiable and contains no stack trace.
Closing event loop: Calling the method close() shuts down an event loop. The method close() first calls the method stop(). The method stop() notifies event loop to stop executing handlers however, this might not happen immediately. It is not expected that event loops can then be restarted.

//Remove event loop el
el.close();

Monitoring the event loop

The MonitorEventLoop thread monitors application threads to make sure event loop latency remains within acceptable bounds. The thread monitors latency by measuring the time the action method of the application event handlers takes to run. Whenever the method runs beyond an acceptable latency limit, MonitorEventLoop prints a stack trace.

Set the monitor event interval with system property MONITOR_INTERVAL_MS from the EventGroup class:

private static final long MONITOR_INTERVAL_MS = Long.getLong("MONITOR_INTERVAL_MS", 100);

Disable the monitor by setting the system property:

disableLoopBlockMonitor=true

Use any stack trace information to improve the design for efficiency.

Recommendations:

Impose an interval of 100ms for every event loop.
Consider adding Jvm.safepoint calls to help identify hotspots in the code.

Event loop Action Handlers

Each event loop services multiple event handlers. The aggressiveness with which any one handler is serviced is influenced by the handler’s priority as well as other activity on the event loop as a whole. If an event handler returns true from action() it biases the event loop to service the same handler again "soon". How soon depends on a variety of factors and the other work the event loop has to do across the other handlers.

Returning true when there is no actual work to do may waste cycles servicing a handler which has nothing to do, at the expense of stealing cycles away from other handlers. Conversely, returning false when there is work to do will effectively increase latency as the event loop will take the "false" as a hint that several other handlers can be serviced ahead of this one.

As a rule of thumb, an action handler should do a certain amount of work then yield/return. If it knows for sure that there is remaining work to be done at the point of yielding then return true. Otherwise return false and the event loop will revisit based on the handler’s priority and other work load. As with a lot of scheduling approaches there’s no single answer and some experimentation under typical loads would always be recommended. But the above rule of thumb is a good starting point.

Pauser

Chronicle Threads provides a number of implementations of the net.openhft.chronicle.threads.Pauser interface.

The recommended way to use Pauser:

    while (running) {}
        // pollForWork returns true if it does something, false if it does nothing
        if (pollForWork())
            pauser.reset();
        else
            pauser.pause();
    }

The various implementations of Pauser allow for varied pausing strategies - see the javadoc.

Pauser modes

For the best performance, the default busy Pauser mode minimises jitter. However, it does maximise CPU usage and CPUs will run hotter. If there are too many threads in busy mode, a machine may slow down.

Table 1. Alternative pauser modes

Mode	Benefits	Downside	monitoring	isolcpus
`busy`	Minimises jitter	Uses more CPU, no monitoring support	☒	☑
`timedBusy`	Minimises jitter	Uses more CPU, slight overhead for monitoring	☑	☑
`yielding`	Low jitter, can be shared	Uses more CPU	☑	☒
`balanced`	Good balance of busy waiting and back off	Uses less CPU, but more jitter	☑	☒
`milli`	Regular checks every 1 ms	Uses minimal CPU, but 1 ms jitter	☑	☒
`sleepy`	Minimal CPU, like balanced but less CPU	More millisecond jitter	☑	☒

For example

In a simple example which is reading from and writing to a socket. The handler typically returns true if anything was read or written on the assumption it may need to read/write something very soon. However, if nothing is read or written, it may still be called soon however this is where the PauserMode determines how the event loop will start backing off when no work is being done.

dkhamitov/Chronicle-Threads