Cloudtoid Interprocess is a cross-platform shared memory queue for fast communication between processes (Interprocess Communication or IPC). It uses a shared memory-mapped file for extremely fast and efficient communication between processes and it is used internally by Microsoft.
- Fast: It is extremely fast.
- Cross-platform: It supports Windows, and Unix-based operating systems such as Linux, MacOS, and FreeBSD.
- API: Provides a simple and intuitive API to enqueue/send and dequeue/receive messages.
- Multiple publishers and subscribers: It supports multiple publishers and subscribers to a shared queue.
- Efficient: Sending and receiving messages is almost heap memory allocation free reducing garbage collections.
- Developer: Developed by a guy at Microsoft.
The NuGet package for this library is published here.
Note: To improve performance, this library only supports 64-bit CLR with 64-bit processor architectures. Attempting to use this library on 32-bit processors, 32-bit operating systems, or on WOW64 may throw a
NotSupportedException
.
This library supports .NET Core 3.1+ and .NET 5+. It is optimized for .NET dependency injection but can also be used without DI.
Creating a message queue factory:
var factory = new QueueFactory();
Creating a message queue publisher:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var publisher = factory.CreatePublisher(options);
publisher.TryEnqueue(message);
Creating a message queue subscriber:
options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var subscriber = factory.CreateSubscriber(options);
subscriber.TryDequeue(messageBuffer, cancellationToken, out var message);
Adding the queue factory to the DI container:
services
.AddInterprocessQueue() // adding the queue related components
.AddLogging(); // optionally, we can enable logging
Creating a message queue publisher using an instance of IQueueFactory
retrieved from the DI container:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var publisher = factory.CreatePublisher(options);
publisher.TryEnqueue(message);
Creating a message queue subscriber using an instance of IQueueFactory
retrieved from the DI container:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var subscriber = factory.CreateSubscriber(options);
subscriber.TryDequeue(messageBuffer, cancellationToken, out var message);
To see a sample implementation of a publisher and a subscriber process, try out the following two projects. You can run them side by side and see them in action:
Please note that you can start multiple publishers and subscribers sending and receiving messages to and from the same message queue.
A lot has gone into optimizing the implementation of this library. For instance, it is mostly heap-memory allocation free, reducing the need for garbage collection induced pauses.
Summary: In average, enqueuing a message is about ~10 ns
and a full enqueue followed by a dequeue takes roughly ~400 ns
on Windows, ~300 ns
on linux, and ~700 ns
.
Details: To benchmark the performance and memory usage, we use BenchmarkDotNet and perform the following runs:
Method | Description |
---|---|
Message enqueue | Benchmarks the performance of enqueuing a message. |
Message enqueue and dequeue | Benchmarks the performance of sending a message to a client and receiving that message. It is inclusive of the duration to enqueue and dequeue a message. |
Message enqueue and dequeue - no message buffer | Benchmarks the performance of sending a message to a client and receiving that message. It is inclusive of the duration to enqueue and dequeue a message and memory allocation for the received message. |
You can replicate the results by running the following command:
dotnet run Interprocess.Benchmark.csproj --configuration Release
You can also be explicit about the .NET SDK and Runtime(s) versions:
dotnet run Interprocess.Benchmark.csproj --configuration Release --framework net5.0 --runtimes net5.0 netcoreapp3.1
Host:
BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19042
Intel Xeon CPU E5-1620 v3 3.50GHz, 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.201
[Host] : .NET Core 5.0.4, X64 RyuJIT
.NET Core 3.1 : .NET Core 3.1.13, X64 RyuJIT
Results:
Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
---|---|---|---|---|
Message enqueue | 7.041 |
0.0753 |
0.0629 |
- |
Message enqueue and dequeue | 390.081 |
3.940 |
3.4930 |
- |
Message enqueue and dequeue - no message buffer | 375.899 |
3.706 |
3.4664 |
32 B |
Host:
OS=macOS Catalina 10.15.6
Intel Core i5-8279U CPU 2.40GHz (Coffee Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.1.401
[Host] : .NET Core 3.1.7, X64 RyuJIT
.NET Core 3.1 : .NET Core 3.1.7, X64 RyuJIT
Results:
Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
---|---|---|---|---|
Message enqueue | 14.19 |
0.05 |
0.04 |
- |
Message enqueue and dequeue | 666.10 |
10.91 |
10.20 |
- |
Message enqueue and dequeue - no message buffer | 689.33 |
13.38 |
15.41 |
32 B |
On Ubuntu (through WSL)
Host:
BenchmarkDotNet=v0.12.1, OS=ubuntu 20.04
Intel Xeon CPU E5-1620 v3 3.50GHz, 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.201
[Host] : .NET Core 5.0.4, X64 RyuJIT
.NET Core 3.1 : .NET Core 3.1.13, X64 RyuJIT
Results:
Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
---|---|---|---|---|
Message enqueue | 13.89 |
0.102 |
0.080 |
- |
Message enqueue and dequeue | 283.55 |
5.592 |
7.839 |
- |
Message enqueue and dequeue - no message buffer | 271.17 |
4.355 |
3.400 |
32 B |
This library relies on [Named Semaphores][NamedSemaphoresDoc] To signal the existence of a new message to all message subscribers and to do it across process boundaries. Named semaphores are synchronization constructs accessible across processes.
.NET Core 3.1 and .NET 5 do not support named semaphores on Unix-based OSs (Linux, macOS, etc.). Instead we are using P/Invoke and relying on operating system's POSIX semaphore implementation. (Linux and MacOS implementations).
This implementation will be replaced with System.Threading.Semaphore
once .NET adds support for named semaphores on all platforms.
- Create a branch from
main
. - Ensure that all tests pass on Windows, Linux, and MacOS.
- Keep the code coverage number above 80% by adding new tests or modifying the existing tests.
- Send a pull request.
Pedram Rezaei is a software architect at Microsoft with years of experience building highly scalable and reliable cloud-native applications for Microsoft.
Here are a couple of items that we are working on.
- Create a documentation website