NEWS: Our paper Occlum: Secure and Efficient Multitasking Inside a Single Enclave of Intel SGX has been accepted by ASPLOS'20. This research paper highlights the advantages of the single-address-space architecture adopted by Occlum and describes a novel in-enclave isolation mechanism that complements this approach. The paper can be found on ACM Digital Library and Arxiv.
Occlum is a memory-safe, multi-process library OS (LibOS) for Intel SGX. As a LibOS, it enables legacy applications to run on SGX with little or even no modifications of source code, thus protecting the confidentiality and integrity of user workloads transparently.
Occlum has the following salient features:
- Efficient multitasking. Occlum offers light-weight LibOS processes: they are light-weight in the sense that all LibOS processes share the same SGX enclave. Compared to the heavy-weight, per-enclave LibOS processes, Occlum's light-weight LibOS processes is up to 1,000X faster on startup and 3X faster on IPC. In addition, Occlum offers an optional multi-domain Software Fault Isolation scheme to isolate the Occlum LibOS processes if needed.
- Multiple file system support. Occlum supports various types of file systems, e.g., read-only hashed FS (for integrity protection), writable encrypted FS (for confidentiality protection), untrusted host FS (for convenient data exchange between the LibOS and the host OS).
- Memory safety. Occlum is the first SGX LibOS written in a memory-safe programming language (Rust). Thus, Occlum is much less likely to contain low-level, memory-safety bugs and is more trustworthy to host security-critical applications.
- Ease-of-use. Occlum provides user-friendly build and command-line tools. Running applications on Occlum inside SGX enclaves can be as simple as only typing several shell commands (see the next section).
If you were to write an SGX Hello World project using some SGX SDK, the project would consist of hundreds of lines of code. And to do that, you have to spend a great deal of time to learn the APIs, the programming model, and the build system of the SGX SDK.
Thanks to Occlum, you can be freed from writing any extra SGX-aware code and only need to type some simple commands to protect your application with SGX transparently---in four easy steps.
Step 1. Compile the user program with the Occlum toolchain (e.g., occlum-gcc)
$ occlum-gcc -o hello_world hello_world.c
$ ./hello_world
Hello World
Note that the Occlum toolchain is not cross-compiling in the traditional sense: the binaries built by the Occlum toolchain is also runnable on Linux. This property makes it convenient to compile, debug, and test user programs intended for Occlum.
Step 2. Initialize a directory as the Occlum instance via occlum init or occlum new
$ mkdir occlum_instance && cd occlum_instance
$ occlum init
or
$ occlum new occlum_instance
The occlum init command creates the compile-time and run-time state of Occlum in the current working directory. The occlum new command does basically the same thing but in a new instance diretory. Each Occlum instance directory should be used for a single instance of an application; multiple applications or different instances of a single application should use different Occlum instances.
Step 3. Generate a secure Occlum FS image and Occlum SGX enclave via occlum build
$ cp ../hello_world image/bin/
$ occlum build
The content of the image directory is initialized by the occlum init command. The structure of the image directory mimics that of an ordinary UNIX FS, containing directories like /bin, /lib, /root, /tmp, etc. After copying the user program hello_world into image/bin/, the image directory is packaged by the occlum build command to generate a secure Occlum FS image as well as the Occlum SGX enclave. The FS image is integrity protected by default, if you want to protect the confidentiality and integrity with your own key, please check out here.
For platforms that don't support SGX, it is also possible to run Occlum in SGX simulation mode. To switch to the simulation mode, occlum build command must be given an extra argument or an environment variable as shown below:
$ occlum build --sgx-mode SIM
or
$ SGX_MODE=SIM occlum build
Step 4. Run the user program inside an SGX enclave via occlum run
$ occlum run /bin/hello_world
Hello World!
The occlum run command starts up an Occlum SGX enclave, which, behind the scene, verifies and loads the associated Occlum FS image, spawns a new LibOS process to execute /bin/hello_world, and eventually prints the message.
Occlum can be configured easily via a configuration file named Occlum.json, which is generated by the occlum init command in the Occlum instance directory. The user can modify Occlum.json to configure Occlum. A sample of Occlum.json is shown below. Some comments are added to provide a brief explanation. If you are not sure how to set the resource_limits or process for your application, please check out Resource Configuration Guide.
{
// Resource limits
"resource_limits": {
// The total size of enclave memory available to LibOS processes
"user_space_size": "256MB",
// The heap size of LibOS kernel
"kernel_space_heap_size": "32MB",
// The stack size of LibOS kernel
"kernel_space_stack_size": "1MB",
// The max number of LibOS threads/processes
"max_num_of_threads": 32
},
// Process
"process": {
// The stack size of the "main" thread
"default_stack_size": "4MB",
// The max size of memory allocated by brk syscall
"default_heap_size": "16MB",
// The max size of memory by mmap syscall (OBSOLETE. Users don't need to modify this field. Keep it only for compatibility)
"default_mmap_size": "32MB"
},
// Entry points
//
// Entry points specify all valid path prefixes for <path> in `occlum run
// <path> <args>`. This prevents outside attackers from executing arbitrary
// commands inside an Occlum-powered enclave.
"entry_points": [
"/bin"
],
// Environment variables
//
// This gives a list of environment variables for the "root"
// process started by `occlum exec` command.
"env": {
// The default env vars given to each "root" LibOS process. As these env vars
// are specified in this config file, they are considered trusted.
"default": [
"OCCLUM=yes"
],
// The untrusted env vars that are captured by Occlum from the host environment
// and passed to the "root" LibOS processes. These untrusted env vars can
// override the trusted, default envs specified above.
"untrusted": [
"EXAMPLE"
]
},
// Enclave metadata
"metadata": {
// Enclave signature structure's ISVPRODID field
"product_id": 0,
// Enclave signature structure's ISVSVN field
"version_number": 0,
// Whether the enclave is debuggable through special SGX instructions.
// For production enclave, it is IMPORTANT to set this value to false.
"debuggable": true,
// Whether to turn on PKU feature in Occlum
// Occlum uses PKU for isolation between LibOS and userspace program,
// It is useful for developers to detect potential bugs.
//
// "pkru" = 0: PKU feature must be disabled
// "pkru" = 1: PKU feature must be enabled
// "pkru" = 2: PKU feature is enabled if the platform supports it
"pkru": 0
},
// Mount points and their file systems
//
// The default configuration is shown below.
"mount": [
{
"target": "/",
"type": "unionfs",
"options": {
"layers": [
{
"target": "/",
"type": "sefs",
"source": "./build/mount/__ROOT",
"options": {
"MAC": ""
}
},
{
"target": "/",
"type": "sefs",
"source": "./run/mount/__ROOT"
}
]
}
},
{
"target": "/host",
"type": "hostfs",
"source": "."
},
{
"target": "/proc",
"type": "procfs"
},
{
"target": "/dev",
"type": "devfs"
}
]
}- Occlum has added several new experimental commands, which provide a more container-like experience to users, as shown below:
occlum init
occlum build
occlum start
occlum exec <cmd1> <args1>
occlum exec <cmd2> <args2>
occlum exec <cmd3> <args3>
occlum stop
- Occlum has enabled per process resource configuration via
prlimitsyscall (https://man7.org/linux/man-pages//man2/prlimit.2.html) and shell built-in commandulimit(https://fishshell.com/docs/current/cmds/ulimit.html). For more info, please read README.md ofdemos/fish.
We have built and tested Occlum on Ubuntu 20.04 with or without hardware SGX support (if the CPU does not support SGX, Occlum can be run in the SGX simulation mode). To give Occlum a quick try, one can use the Occlum Docker image by following the steps below:
Step 1-3 are to be done on the host OS (Linux):
-
For Linux kernel before 5.11, Intel SGX driver for Linux must be installed manually.
-
For Linux kernel before 5.9, install enable_rdfsbase kernel module, which enables Occlum to use
rdfsbase-family instructions in enclaves. -
Run the Occlum Docker container, which has Occlum and its demos preinstalled:
For old IAS driver (not DCAP aware):
docker run -it --device /dev/isgx occlum/occlum:[version]-ubuntu20.04
For DCAP driver before v1.41:
docker run -it --device /dev/sgx/enclave --device /dev/sgx/provision occlum/occlum:[version]-ubuntu20.04
For DCAP driver since v1.41 or in-tree kernel driver:
# Two methods: # (1) Create softlinks on host mkdir -p /dev/sgx cd /dev/sgx && ln -sf ../sgx_enclave enclave && ln -sf ../sgx_provision provision docker run -it --device /dev/sgx/enclave --device /dev/sgx/provision occlum/occlum:[version]-ubuntu20.04 # (2) Create the docker with privileged mode docker run -it --privileged -v /dev/sgx_enclave:/dev/sgx/enclave -v /dev/sgx_provision:/dev/sgx/provision occlum/occlum:[version]-ubuntu20.04
Step 4-5 are to be done on the guest OS running inside the Docker container:
- (Optional) Try the sample code of Intel SGX SDK to make sure that SGX is working
cd /opt/intel/sgxsdk/SampleCode/SampleEnclave && make && ./app - Check out Occlum's demos preinstalled at
/root/demos, whose README can be found here. Or you can try to build and run your own SGX-protected applications using Occlum as shown in the demos.
Alternatively, to use Occlum without Docker, one can install Occlum on popular Linux distributions like Ubuntu and CentOS with the Occlum DEB and RPM packages, respectively. These packages are provided for every release of Occlum since 0.16.0. For more info about the packages, see here.
There are many different projects to demonstrate how Occlum can be used to build and run user applications, which could be found on demos.
There is also a whole-flow confidential inference service example to demonstrate how to convert a real application directly from Docker image to Occlum image, how to integrate Occlum Init-RA solution for whole-flow sensitive data protection plus how to generate and run the Docker container based Occlum instances.
Moreover, an example to deploy Occlum Confidential Inference Service on Kubernetes can be found here.
To build Occlum from the latest source code, do the following steps in an Occlum Docker container (which can be prepared as shown in the last section):
-
Download the latest source code of Occlum
mkdir occlum && cd occlum git clone https://github.com/occlum/occlum . -
Prepare the submodules and tools required by Occlum.
make submodule -
Compile and test Occlum
make # test musl based binary make test # test glibc based binary make test-glibc # stress test make test times=100For platforms that don't support SGX
SGX_MODE=SIM make SGX_MODE=SIM make test -
Install Occlum
make installwhich will install the
occlumcommand-line tool and other files at/opt/occlum.
The Occlum Dockerfile can be found at here. Use it to build the container directly or read it to see the dependencies of Occlum.
Occlum supports running any executable binaries that are 1) based on musl libc and 2) position independent. We chose musl libc instead of Glibc since the codebase of musl libc is 10X smaller than Glibc, which means a much smaller Trusted Computing Base (TCB) and attack surface. We argue this is an important consideration for Occlum, which targets security-critical apps running inside SGX enclaves.
The two aforementioned requirements are not only satisfied by the Occlum toolchain, but also the native toolchains from some Linux distributions, e.g., Alpine Linux. We think Alpine Linux, a popular Linux distribution that emphasizes simplicity and security, is a natural fit for Occlum. We have provided demos (see Python) to run unmodified apps from Alpine Linux packages.
Since 0.19.0, Occlum added Glibc support. Thus, now users can run executables from popular Linux distributions (e.g., Ubuntu) on Occlum without recompiling them.
To debug an app running upon Occlum, one can harness Occlum's builtin support for GDB via occlum gdb command. More info can be found here.
Meanwhile, one can use occlum mount command to access and manipulate the secure filesystem for debug purpose. More info can be found here.
If the cause of a problem does not seem to be the app but Occlum itself, then one can take a glimpse into the inner workings of Occlum by checking out its log. Occlum's log level can be adjusted through OCCLUM_LOG_LEVEL environment variable. It has six levels: off, error, warn, debug, info, and trace. The default value is off, i.e., showing no log messages at all. The most verbose level is trace.
By default, the occlum build command builds and signs enclaves in debug mode. These SGX debug-mode enclaves are intended for development and testing purposes only. For production usage, the enclaves must be signed by a key acquired from Intel (a restriction that will be lifted in the future when Flexible Launch Control is ready) and run with SGX debug support disabled.
Occlum has built-in support for both building and running enclaves in release mode.
To do that, modify Occlum.json [metadata]-[debuggable] field to false. And then run the commands below:
$ occlum build --sign-key <path_to/your_key.pem>
$ occlum run <prog_path> <prog_args>
Ultimately, whether an enclave is running in the release mode should be checked and judged by a trusted client through remotely attesting the enclave. See the remote attestation demo here.
To cut off the complexity of self-hosted infrastructure, one can deploy Occlum-powered SGX apps on public clouds with SGX support. For example, we have tested and successfully deployed Occlum Docker containers on Azure Kubernetes Service (AKS). Please check out this doc for more details.
Occlum is being actively developed. We now focus on implementing more system calls and additional features required in the production environment.
While this project is still not mature or stable (we are halfway through reaching version 1.0.0), we have used Occlum to port many real-world applications (like Tensorflow Lite, XGBoost, GCC, Lighttpd, etc.) to SGX with little or no source code modifications. We believe that the current implementation of Occlum is already useful to many users and ready to be deployed in some use cases.
The high-level architecture of Occlum is summarized in the figure below:
The project name Occlum stems from the word Occlumency coined in Harry Potter series by J. K. Rowling. In Harry Potter and the Order of Phoenix, Occlumency is described as:
The magical defence of the mind against external penetration. An obscure branch of magic, but a highly useful one... Used properly, the power of Occlumency will help shield you from access or influence.
The same thing can be said for Occlum, not for the mind, but for the program:
The magical defence of the program against external penetration. An obscure branch of technology, but a highly useful one... Used properly, the power of Occlum will help shield your program from access or influence.
Of course, Occlum must be run on Intel x86 CPUs with SGX support to do its magic.
This project follows the all-contributors specification. Contributions of any kind are welcome! We will publish contributing guidelines and accept pull requests after the project gets more stable.
Thanks go to all these wonderful contributors to this project.
Occlum is released by Ant Financial under BSD License. See the copyright information here.