Rust library for building linux ublk target device, which talks with
linux ublk driver1 for exposing standard linux block device,
meantime all target IO logic is implemented in userspace.
Linux kernel 6.0 starts to support ublk covered by config option of CONFIG_BLK_DEV_UBLK.
Follows one 2-queue ublk-null target which is built over libublk, ublk block device(/dev/ublkbN) is created after the code is run. And the device will be deleted after terminating this process by ctrl+C.
use libublk::{ctrl::UblkCtrlBuilder, io::UblkDev, io::UblkQueue};
// async/.await IO handling
async fn handle_io_cmd(q: &UblkQueue<'_>, tag: u16) -> i32 {
(q.get_iod(tag).nr_sectors << 9) as i32
}
// implement whole ublk IO level protocol
async fn io_task(q: &UblkQueue<'_>, tag: u16) {
// IO buffer for exchange data with /dev/ublkbN
let buf_bytes = q.dev.dev_info.max_io_buf_bytes as usize;
let buf = libublk::helpers::IoBuf::<u8>::new(buf_bytes);
let mut cmd_op = libublk::sys::UBLK_U_IO_FETCH_REQ;
let mut res = 0;
// Register IO buffer, so that buffer pages can be discarded
// when queue becomes idle
q.register_io_buf(tag, &buf);
loop {
// Complete previous command with result and re-submit
// IO command for fetching new IO request from /dev/ublkbN
res = q.submit_io_cmd(tag, cmd_op, buf.as_mut_ptr(), res).await;
if res == libublk::sys::UBLK_IO_RES_ABORT {
break;
}
// Handle this incoming IO command
res = handle_io_cmd(&q, tag).await;
cmd_op = libublk::sys::UBLK_U_IO_COMMIT_AND_FETCH_REQ;
}
}
fn q_fn(qid: u16, dev: &UblkDev) {
let q_rc = std::rc::Rc::new(UblkQueue::new(qid as u16, &dev).unwrap());
let exe = smol::LocalExecutor::new();
let mut f_vec = Vec::new();
for tag in 0..dev.dev_info.queue_depth {
let q = q_rc.clone();
f_vec.push(exe.spawn(async move { io_task(&q, tag).await }));
}
// Drive smol executor, won't exit until queue is dead
libublk::uring_async::ublk_wait_and_handle_ios(&exe, &q_rc);
smol::block_on(async { futures::future::join_all(f_vec).await });
}
fn main() {
// Create ublk device
let ctrl = std::sync::Arc::new(
UblkCtrlBuilder::default()
.name("async_null")
.nr_queues(2)
.dev_flags(libublk::UblkFlags::UBLK_DEV_F_ADD_DEV)
.build()
.unwrap(),
);
// Kill ublk device by handling "Ctrl + C"
let ctrl_sig = ctrl.clone();
let _ = ctrlc::set_handler(move || {
ctrl_sig.kill_dev().unwrap();
});
// Now start this ublk target
ctrl.run_target(
// target initialization
|dev| {
dev.set_default_params(250_u64 << 30);
Ok(())
},
// queue IO logic
|tag, dev| q_fn(tag, dev),
// dump device after it is started
|dev| dev.dump(),
)
.unwrap();
// Usually device is deleted automatically when `ctrl` drops, but
// here `ctrl` is leaked by the global sig handler closure actually,
// so we have to delete it explicitly
ctrl.del_dev().unwrap();
}-
examples/loop.rs: real example using async/await & io_uring. -
examples/ramdisk.rs: single thread & async/.await for both ctrl and IO, this technique will be extended to create multiple devices from single thread in future
rublk2 is based on libublk, and supports null, loop, zoned & qcow2 targets so
far.
In unprivileged mode(UBLK_F_UNPRIVILEGED_DEV), ublk device can be created
in non-admin user session. For supporting this feature:
- install udev rules
KERNEL=="ublk-control", MODE="0666", OPTIONS+="static_node=ublk-control"
ACTION=="add",KERNEL=="ublk[bc]*",RUN+="/usr/local/sbin/ublk_chown.sh %k 'add' '%M' '%m'"
ACTION=="remove",KERNEL=="ublk[bc]*",RUN+="/usr/local/sbin/ublk_chown.sh %k 'remove' '%M' '%m'"
- install utility and script
utils/ublk_chown.sh and binary of utils/ublk_user_id.rs needs to be
installed under /usr/local/sbin or other directory which has to match
with the udev rules.
You can run the test of the library with cargo test
When running fio t/io_uring /dev/ublkb03, IOPS is basically same with
running same test over ublk device created by blktests miniublk4, which
is written by pure C. And the ublk device is null, which has 2 queues, each
queue's depth is 64.
cargo run --example loop help
cargo run --example null help
This project is licensed under either of Apache License, Version 2.0 or MIT license at your option.
Any kinds of contributions are welcome!