Clojure native executable using GraalVM. 100x Speedup!
GraalVM is an alternative compiler which can compile Clojure (and many other languages) into a native, statically-linked executable. This executable runs with minimal memory and minimum startup time, just like a C/C++ version of "Hello, World!".
The GraalVM distribution is a full OpenJDK-11 distribution. I always install packages like
this in /opt
, so the final install dir looks like:
/opt/graalvm-ce-java11-19.3.0
Download the GraalVM tar file from the
the GraalVM Releases page. Unpack the tar
file and install it in /opt
:
~ > alias d='ls -ldF'
~ > cd ~/Downloads
~/Downloads > d graalvm*
-rw-rw-r-- 1 alan alan 465117693 Nov 26 16:48 graalvm-ce-java11-linux-amd64-19.3.0.tar.gz
~/Downloads > mkdir -p tmp; cd tmp
~/Downloads/tmp > ls -al
total 0
drwxr-xr-x 2 r634165 RBSWA\Domain Users 64 Nov 6 13:42 ./
drwx------+ 16 r634165 RBSWA\Domain Users 512 Nov 8 11:44 ../
~/Downloads/tmp > tar -xf ../graalvm-ce-java11-linux-amd64-19.3.0.tar.gz
~/Downloads/tmp > d *
drwxr-xr-x 3 r634165 RBSWA\Domain Users 96 Nov 8 11:45 graalvm-ce-19.3.0/
~/Downloads/tmp > sudo mkdir -p /opt
Password:
~/Downloads/tmp > sudo mv graalvm-ce-19.2.1 /opt
~/Downloads/tmp > cd /opt
/opt > d graal*
drwxr-xr-x 3 r634165 RBSWA\Domain Users 96 Nov 8 11:45 graalvm-ce-19.3.0/
# I like to create a symbolic link so our ~/.bashrc file doesn't need to change when we upgrade graalvm versions
/opt > sudo ln -s graalvm-ce-19.2.1 graalvm
> d /opt/graal*
lrwxr-xr-x 1 root wheel 17 Nov 8 09:56 /opt/graalvm@ -> graalvm-ce-19.3.0
drwxr-xr-x 3 r634165 RBSWA\Domain Users 96 Nov 6 13:42 /opt/graalvm-ce-19.3.0/
I have the following basic setup in ~/.bashrc
# Utility functions to ease PATH-building syntax
function path_prepend() {
local path_search_dir=$1
export PATH="${path_search_dir}:${PATH}"
}
function path_append() {
local path_search_dir=$1
export PATH="${PATH}:${path_search_dir}"
}
# Basic PATH
export PATH=.
path_append ${HOME}/bin
path_append ${HOME}/opt/bin
path_append /opt/bin
path_append /usr/local/bin
path_append /usr/bin
path_append /bin
function graalvm() {
export JAVA_HOME=/opt/graalvm # linux version *** PICK ONE ***
export JAVA_HOME=/opt/graalvm/Contents/Home # OSX version *** PICK ONE ***
path_prepend ${JAVA_HOME}/bin
java -version
}
function java13() {
export JAVA_HOME=$(/usr/libexec/java_home -v 13) # Mac OSX java trick
path_prepend ${JAVA_HOME}/bin
java -version
}
java13 >& /dev/null # set java13 to be default
Verify you can switch your environment from the default Java (here, jdk13) to GraalVM’s version of OpenJDK-8:
~/expr/graalvm > java -version # when login, we were using jdk13
java version "13" 2019-09-17
Java(TM) SE Runtime Environment (build 13+33)
Java HotSpot(TM) 64-Bit Server VM (build 13+33, mixed mode, sharing)
~/expr/graalvm > graalvm # now, switch to GraalVM (jdk8)
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-20191009173705.graal.jdk8u-src-tar-gz-b07)
OpenJDK 64-Bit GraalVM CE 19.2.1 (build 25.232-b07-jvmci-19.2-b03, mixed mode)
> gu --help
GraalVM Component Updater v2.0.0
Usage:
...<snip>...
gu install [-0CcDfiLnosruvyxY] <param> installs a component package
...<snip>...
~/expr/graalvm > time lein do clean, run
Hello, World!
Goodbye...
lein do clean, run 9.92s user 0.84s system 225% cpu 4.780 total
~/expr/graalvm > time lein uberjar
Compiling hello-world.core
Created /Users/r634165/expr/graalvm/target/hello-world-0.1.0-SNAPSHOT.jar
Created /Users/r634165/expr/graalvm/target/hello-world-0.1.0-SNAPSHOT-standalone.jar
lein uberjar 11.21s user 2.71s system 182% cpu 7.626 total
~/expr/graalvm > time java -jar target/hello-world-0.1.0-SNAPSHOT-standalone.jar
Hello, World!
Goodbye...
java -jar target/hello-world-0.1.0-SNAPSHOT-standalone.jar 2.67s user 0.26s system 226% cpu 1.297 total
So, it took 7.6 sec to compile and package the uberjar, and 1.3 seconds to run the uberjar.
~/expr/graalvm > lein native
Build on Server(pid: 59523, port: 58080)*
[./target/hello-world:59523] classlist: 2,895.07 ms
[./target/hello-world:59523] (cap): 1,955.86 ms
[./target/hello-world:59523] setup: 3,245.68 ms
[./target/hello-world:59523] (typeflow): 4,537.50 ms
[./target/hello-world:59523] (objects): 2,574.54 ms
[./target/hello-world:59523] (features): 276.47 ms
[./target/hello-world:59523] analysis: 7,572.88 ms
[./target/hello-world:59523] (clinit): 146.73 ms
[./target/hello-world:59523] universe: 436.47 ms
[./target/hello-world:59523] (parse): 528.53 ms
[./target/hello-world:59523] (inline): 1,580.97 ms
[./target/hello-world:59523] (compile): 5,630.39 ms
[./target/hello-world:59523] compile: 8,228.69 ms
[./target/hello-world:59523] image: 875.32 ms
[./target/hello-world:59523] write: 558.38 ms
[./target/hello-world:59523] [total]: 24,045.25 ms
The GraalVM compiler is similar to the Google Closure compiler used to make GMail, etc super-compact & lightning-fast to
download & run over the internet. Besides compiling the source code, it performs a static analysis to eliminate all
unreachable code, in addition to normal optimization steps. This results in a minimal executable size, and the
fast startup we expect from a statically linked executable (for example, the ls
command).
~/expr/graalvm > time target/hello-world
Hello, World!
Goodbye...
target/hello-world 0.00s user 0.00s system 52% cpu 0.009 total
Yes, you read that right! Instead of taking 1.3 seconds to run the uberjar, we needed less than 0.01 seconds to run the native executable, for a speedup of over 130x !
Just for fun, let’s compare to the ls
command:
~/expr/graalvm > time ls -ldF *
-rw-r--r-- 1 r634165 RBSWA\Domain Users 14199 Nov 6 13:51 LICENSE
-rw-r--r-- 1 r634165 RBSWA\Domain Users 7126 Nov 8 12:47 README.adoc
drwxr-xr-x 3 r634165 RBSWA\Domain Users 96 Nov 8 10:48 doc/
-rw-r--r-- 1 r634165 RBSWA\Domain Users 1528 Nov 7 10:57 hello-world.iml
-rw-r--r-- 1 r634165 RBSWA\Domain Users 657 Nov 7 10:56 project.clj
drwxr-xr-x 2 r634165 RBSWA\Domain Users 64 Nov 6 13:51 resources/
drwxr-xr-x 3 r634165 RBSWA\Domain Users 96 Nov 6 13:51 src/
drwxr-xr-x 7 r634165 RBSWA\Domain Users 224 Nov 8 12:06 target/
ls -ldF * 0.00s user 0.00s system 61% cpu 0.010 total
This command required 0.01 seconds, and it is apparent that Clojure+GraalVM has achieved parity with command-line utilities written in C.
Note that using time
as above resolves to a shell built-in command. We can get more information
from the standard Unix version of time
:
# JVM+UberJar
> /usr/bin/time -l java -jar target/hello-world-0.1.0-SNAPSHOT-standalone.jar
Hello, World!
Goodbye...
1.20 real 2.47 user 0.24 sys
409 maximum resident set size (MB)
100469 page reclaims
3569 involuntary context switches
# Static Executable
> /usr/bin/time -l target/hello-world
Hello, World!
Goodbye...
0.00 real 0.00 user 0.00 sys
2 maximum resident set size (MB)
657 page reclaims
4 involuntary context switches
So we see that the maximum RSS memory requirement was reduced from 409 Mb to 2 Mb. Yes, an improvement over 200x! Note also that context switches have been reduced by 900x, and page reclaims by about 200x.
Here is a quick comparison with Python:
> time python -c 'print("Hello world!")'
Hello world!
0.03s user 0.01s system 80% cpu 0.048 total
> /usr/bin/time -l python -c 'print("Hello world!")'
Hello world!
0.04 real 0.02 user 0.01 sys
6 maximum resident set size (MB)
2110 page reclaims
24 involuntary context switches
So the Python version takes 5x longer, and uses 3x more memory.
Anywhere you want to use your favorite language in a constrained environment, where startup speed and/or memory usage is a concern. Obvious use-cases include command-line utilities and cloud serverless functions such as AWS Lambda.
-
Nice ClojureD video by Jan Stepien
-
Bruno Bonacci’s GraalVM Clojure Demo
-
The GraalVM Project Homepage
-
GraalVM Downloads
I use additional bash functions to help config. Namely:
function isMac() {
if [[ $(uname -a) =~ "Darwin" ]]; then
true
else
false
fi
}
function isLinux() {
if [[ $(uname -a) =~ "Linux" ]]; then
true
else
false
fi
}
This allows a single .bashrc file to control both linux & Mac computers as follows:
if $(isLinux) ; then #{
# echo "Found Linux"
function java13() {
export JAVA_HOME=/opt/java13
path_prepend "${JAVA_HOME}/bin"
java --version
}
function graalvm() {
export JAVA_HOME=/opt/graalvm
path_prepend ${JAVA_HOME}/bin
java -version
}
fi #}
if $(isMac) ; then #{
# echo "Found Darwin (block)"
function java13() {
export JAVA_HOME=$(/usr/libexec/java_home -v 13)
path_prepend ${JAVA_HOME}/bin
java -version
}
function graalvm() {
export JAVA_HOME=/opt/graalvm/Contents/Home
path_prepend ${JAVA_HOME}/bin
java -version
}
fi #}
java13 >& /dev/null # ********** default java version to use **********
alias d=' ls -ldF --color'
alias lal=' ls -alF --color'
Copyright © 2019 Alan Thompson
This program and the accompanying materials are made available under the terms of the Eclipse Public License 2.0 which is available at http://www.eclipse.org/legal/epl-2.0.
This Source Code may also be made available under the following Secondary Licenses when the conditions for such availability set forth in the Eclipse Public License, v. 2.0 are satisfied: GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version, with the GNU Classpath Exception which is available at https://www.gnu.org/software/classpath/license.html.