Ocolos is the first online code layout optimization system for unmodified applications written in unmanaged languages. Ocolos allows profile-guided optimization to be performed on a running process, instead of being performed offline and requiring the application to be re-launched. A description of how we implemented Ocolos and experimental results on MySQL-sysbench workloads are in MICRO'22 paper.
For the demonstration purpose, we integrate MySQL
and sysbench
to Ocolos, so this version of Ocolos ONLY works with MySQL
.
Please refer instructions from links or directly run commands listed below to install prerequisites:
- Clang(for building MySQL): https://clang.llvm.org/get_started.html
- Linux Perf:
sudo apt-get install linux-tools-common linux-tools-generic
- libunwind: https://github.com/libunwind/libunwind
- Rust:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
- boost_1_73_0 (for MySQL): https://www.boost.org/users/history/version_1_73_0.html
- binutils (for objdump) 1 : https://ftp.gnu.org/gnu/binutils/
$ git clone git@github.com:upenn-acg/Ocolos-MySQL.git
To use llvm-bolt
and perf2bolt
utilities, BOLT
needs to be installed.
Please follow the commands below to install BOLT
> git clone https://github.com/facebookincubator/BOLT llvm-bolt
> cd llvm-bolt
> git checkout 88c70afe9d388ad430cc150cc158641701397f70
> cp {path to ocolos_mysql directory}/BOLT_replace/RewriteInstance.cpp bolt/lib/Rewrite/RewriteInstance.cpp
> cp {path to ocolos_mysql directory}/BOLT_replace/BinaryContext.cpp bolt/lib/Core/BinaryContext.cpp
> cd .. && mkdir build && cd build
> cmake -G Ninja ../llvm-bolt/llvm -DLLVM_TARGETS_TO_BUILD="X86;AArch64" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS="clang;lld;bolt"
> ninja
> git clone https://github.com/mysql/mysql-server.git
> cd mysql-server
> git checkout 6846e6b2f72931991cc9fd589dc9946ea2ab58c9
In CMakeList.txt
, at line 582, please add:
STRING_APPEND(CMAKE_C_FLAGS " -fno-jump-tables")
STRING_APPEND(CMAKE_CXX_FLAGS " -fno-jump-tables")
Then build mysqld
2 from source:
> export CC={the path of the Clang binary}
> export CXX={the path of the Clang++ binary (actually symbolic link)}
> mkdir build && cd build
> cmake .. -DWITH_BOOST={path of the boost_1_73_0 directory} -DCMAKE_CXX_LINK_FLAGS=-Wl,--emit-relocs -DCMAKE_C_LINK_FLAGS=-Wl,--emit-relocs -DBUILD_CONFIG=mysql_release
> make -j
> make install
To initialize MySQL, run:
> chown {user}:{user} {path to MySQL directory}
> {path to MySQL directory}/bin/mysqld --initialize-insecure --user=root --datadir={your data dir path of MySQL}
> {path to MySQL directory}/bin/mysqld --user=root --port=3306 --datadir={your data dir path of MySQL}
In another terminal, run:
> mysql -u root
> CREATE USER 'ocolos'@'localhost';
> GRANT ALL PRIVILEGES ON *.* TO 'ocolos'@'localhost' WITH GRANT OPTION;
> CREATE DATABASE ocolos_db;
> QUIT;
Note:
- {path to MySQL directory} is normally
/usr/local/mysql
unless otherwise specified during MySQL server's installation. - {user} should be your linux user name.
Please refer instructions in the following webpage:
https://github.com/akopytov/sysbench
- Navigate to
ocolos_mysql
directory. - In the file
config
, specify the absolute path fornm
,perf
,objdump
,llvm-bolt
,perf2bolt
3 - In
config
, please also specify the commands to runMySQL server
andsysbench
. The example commands are given in the config file.- Note: the first argument of the command (a.k.a. the binary being invoked in the command) should be written in its full path.
- Then run the following commands:
> make
> ./extract_call_sites
> ./tracer
make
will produce 2 executables (tracer
&extract_call_sites
)+ 1 shared library (replace_function.so
).- If libunwind library is stored in other places instead of
/usr/local/lib
, you also need to edit Makefile and update it to the corresponding path. - If libunwind's header files are stored in other places instead of
/usr/local/include
, you also need to edit Makefile and update it to the corresponding path.
- If libunwind library is stored in other places instead of
./extract_call_sites
will produce to 2 files which store all call sites information extracted from the target binary (a.k.a.mysqld
) to thetmp_data_dir
you specified in the config file../tracer
will invoke bothMySQL
server process and sysbench workloadsoltp_read_only
, and then perform code layout optimization during runtime.- The output of sysbench's throughput can be found in
sysbench_output.txt
. At about the 130th second, you will see a significant throughput improvement, since Ocolos has replace the code layout to be the optimized one at that time. - After one run (~3 minutes), if you want to start another run, please first run
mysqladmin -u root shutdown
command to shutdown the currentMySQL
server process.
- The output of sysbench's throughput can be found in
In src/utils.hpp
,
- if
TIME_MEASUREMENT
is defined, Ocolos will print the execution time of code replacement; - if
MEASUREMENT
is defined, Ocolos will print metrics such as:- the number of functions on the call stack when target process is paused,
- the number of functions that are moved by BOLT,
- the number of functions that are in the BOLT and original functions.
- if
DEBUG_INFO
is defined, Ocolos will print debug information such as:- the information about detailed behavior of tracer
- the content in the call stack when the target process is paused
DEBUG_INFO
can also be defined insrc/replace_function.hpp
. In this way, the ld_preload library can store all machine code per function it inserted to the target process as auint8_t
format array into a file. The file can be found in thetmp_data_path
you defined in the config file.
- if
DEBUG
is defined, after code replacement, Ocolos will first sendSIGSTOP
signal to target process and then resume the target process byPTRACE_DETACH
. In this way, it allows debugging tools such asGDB
to attach to the target process and observe what goes wrong after code replacement.
Footnotes
-
If your
objdump
version is older than 2.27, please download the latest version ofbinutils
. ↩ -
The target binary must to be compiled by
Clang
rather thangcc
. Agcc
-compiled binary will not work withOcolos
. ↩ -
if
nm
,objdump
andperf
are already in shell, it's OK that their paths are not specified inconfig
. This can be checked bywhich nm
,which objdump
andwhich perf
. ↩