glommio
Join our Zulip community!
If you are interested in Glommio consider joining our Zulip community. Come tell us about exciting applications you are building, ask for help, or really just chat with friends
What is Glommio?
Glommio (pronounced glo-mee-jow or |glomjəʊ|) is a Cooperative Thread-per-Core crate for
Rust & Linux based on io_uring
. Like other rust asynchronous crates it allows
one to write asynchronous code that takes advantage of rust async
/await
, but
unlike its counterparts it doesn't use helper threads anywhere.
Using Glommio is not hard if you are familiar with rust async. All you have to do is:
use glommio::prelude::*;
LocalExecutorBuilder::new().spawn(|| async move {
/// your code here
}).unwrap();
Please note Glommio requires at least 512 KiB of locked memory for io_uring
to work. You can
increase the memlock
resource limit (rlimit) as follows:
$ vi /etc/security/limits.conf
* hard memlock 512
* soft memlock 512
To make the new limits effective, you need to login to the machine again. You can verify that the limits are updated by running the following:
$ ulimit -l
512
For more details check out our docs page and an introductory article
Status
Glommio is still considered an alpha release. The main reasons are:
- The existing API is still evolving
- There are still some uses of unsafe that can be avoided
- There are features that are critical for a good thread per core system
that are not implemented yet. The top one being:
- per-shard memory allocator.
Want to help bring us to production status sooner? PRs are welcome!
Current limitations
Due to our immediate needs which are a lot narrower, we make the following design assumptions:
-
NVMe. While other storage types may work, the general assumptions made in here are based on the characteristics of NVMe storage. This allows us to use io uring's poll ring for reads and writes which are interrupt free. This also assumes that one is running either
XFS
orExt4
(an assumption that Seastar also makes). -
A corollary to the above is that the CPUs are likely to be the bottleneck, so this crate has a CPU scheduler but lacks an I/O scheduler. That, however, would be a welcome addition.
-
A recent(at least 5.8) kernel is no impediment, as long as a fully functional I/O uring is present. In fact, we require a kernel so recent that it doesn't even exist: operations like
mkdir, ftruncate
, etc which are not present in today's (5.8)io_uring
are simply synchronous and we'll live with the pain in the hopes that Linux will eventually add support for them.
Missing features
There are many. In particular:
-
Memory allocator: memory allocation is a big source of contention for thread per core systems. A shard-aware allocator would be crucial for achieving good performance in allocation-heavy workloads.
-
As mentioned, an I/O Scheduler.
-
Visibility: the crate exposes no metrics on its internals, and that should change ASAP.
License
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.