Rkt-io direct userspace network and storage I/O stack specifically designed for TEEs that combines high-performance, POSIX compatibility and security. rkt-io achieves high I/O performance by employing direct userspace I/O libraries (DPDK and SPDK) inside the TEE for kernel-bypass I/O. For efficiency, rkt-io polls for I/O events directly interacting with the hardware instead of relying on interrupts, and it avoids data copies by mapping DMA regions in the untrusted host memory. To maintain full Linux ABI compatibil ity, the userspace I/O libraries are integrated with userspace versions of the Linux VFS and network stacks inside the TEE. Since the I/O stack runs entirely within the TEE, thus omitting the host OS from the I/O path, rkt-io can transparently encrypt all I/O data and does not suffer from host interface/Iago attacks.
Rkt-io was also published in Eurosys 2021. See bibtex for citation.
rkt-io is a fork of sgx-lkl. For normal usage see the old README. For reproducing the paper results read the next headline.
Due its special hardware requirments we provide ssh access to our evaluation machines. Please contact the paper author email address to obtain ssh keys. The machines will have the correct hardware and also software installed to run the experiments. If you run into problems you can write join the IRC channel #rkt-io on freenode fro a live chat (there is also a webchat version at https://webchat.freenode.net/) or write an email for further questions.
- Intel NIC supported by i40e driver: In rkt-io we performed some driver optimizations that required some refactorings in DPDK to reduce memory copy. Hence we had to modify the low-level i40e intel NIC driver. We did not apply those refactorings to other drivers. Hence one needs the same hardware to reproduce the paper results. Our NIC was XL710
- NVME block device: We need a free NVME block device. During evaluation this device will be reformated. We used an Intel DC P4600 2TB NVME drive.
- Intel CPU with SGX support: Most new consumer CPUs have SGX support. Some server xeon processors don't
- A second machine acting as a client. This one needs a similar capable NIC (i.e. same bandwidth). The other machine does not need to have an NVME drive. The second machine must be reachable via ssh.
- Operating system: Linux
- Nix: For reproducibility we use the nix package manager to download all build dependencies. We locked the package versions to ensure reproducibility so that.
- Python 3.7 or newer for the script that reproduces the evaluation
The first step is to get the source code for rkt-io:
$ git clone https://github.com/Mic92/rkt-ioFor convience we created an evaluation script (reproduce.py) that will first build rkt-io and
than run all evaluation experiments from the paper.
This script only depends on Python and Nix as referenced above.
All other dependencies will be loaded through nix.
If the script fails at any point it can be restarted and it will
only not yet done builds or experiments.
Each command it runs will be printed to during evaluation along with
environment variable set.
In addition to some default settings also machine specific settings are
required. The script read those from a file containing the hostname of the
machine + .env. An example configuration file is provided in the repo
(martha.env - @eurosys testers - you don't need to change anything).
To run the evaluation script use the following command:
$ cd rkt-io
$ python reproduce.py After the build is finished, it will start evaluations and generate graphs for each afterwards. The graphs will be written to ./results.
The following figures are reproduced:
-
Figure 1. Micro-benchmarks to showcase the performance of syscalls, storage and network stacks across different systems
- a) System call latency with sendto()
- b) Storage stack performance with fio
- c) Network stack performance with iPerf
-
Figure 5. Micro-benchmarks to showcase the effectiveness of various design choices in rkt-io Effectiveness of the SMP design w/ fio with increasing number of threads
- a) Effectiveness of the SMP design w/ fio with increasing number of threads
- b) iPerf throughput w/ different optimizations
- c) Effectiveness of hardware-accelerated crypto routines
-
Figure 7. The above plots compare the performance of four real-world applications (SQlite, Ngnix, Redis, and MySQL) while running atop native linux
- a) SQLite throughput w/ Speedtest (no security) and three secure systems: Scone, SGX-LKL and rkt-io
- b) Nginx latency w/ wrk
- c) Nginx throughput w/ wrk
- d) Redis throughput w/ YCSB (A)
- e) Redis latency w/ YCSB (A)
- f) MySQL OLTP throughput w/ sys-bench
We published a paper with all technical details about Rkt-IO in Eurosys 2021
@inproceedings{10.1145/3447786.3456255,
author = {J\"{o}rg Thalheim and Harshavardhan Unnibhavi and Christian Priebe and Pramod Bhatotia and Peter Pietzuch},
title = {Rkt-Io: A Direct I/O Stack for Shielded Execution},
booktitle = {Proceedings of the Sixteenth European Conference on Computer Systems},
year = {2021},
}