HALucinator - Firmware rehosting through abstraction layer modeling.

Setup in a Docker

See wiki for instructions for building internal to Sandia

Clone repos into following locations

git clone git@github.com:halucinator/halucinator.git
git clone git@github.com:halucinator/avatar2.git halucinator/deps/avatar2
git clone git@github.com:halucinator/avatar-qemu.git halucinator/deps/avatar2/targets/src/avatar-qemu

Then run Building the docker will take a long time

docker build -t halucinator ./
docker run --name halucinator --rm -i -t halucinator bash
#Inside Docker container run
hal_dev_uart -i=1073811456

In separate terminal run

docker exec -it halucinator bash
#Inside docker container run
./test/STM32/example/run.sh

You will eventually see in both terminals messages containing

 ****UART-Hyperterminal communication based on IT ****
 Enter 10 characters using keyboard :

Enter 10 Characters in the first terminal running hal_dev_uart press enter should then see text echoed followed by.

 Example Finished

Setup in Virtual Environment

Note: This has been lightly tested on Ubuntu 18.04 and 20.04

Install dependencies using ./install_deps.sh
Create and activate a python3 virtual environment (I use virtualmachine wrapper but it is not required). You often have to restart you terminal after installing virutalmachine wrapper for below to work
```
   mkvirtualenv -p `which python3` halucinator
```
If (halucinator) is not in your prompt use workon halucinator

Note: On ubuntu 18.04 you may have to manually configure virtualenvwrapper. Or build you virtual environment using you preferred method
```
    pip3 install virtualenvwrapper
```
Then add to ~/.bashrc using your favorite editor and then run source ~/.bashrc. Replace your username in below
```
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_VIRTUALENV=/home/<your username>/.local/bin/virtualenv
source ~/.local/bin/virtualenvwrapper.sh
```

Get and install Avatar

cd deps
git clone AVATAR_REPO
cd <halucinator_root>/avatar2
pip install -e .

Get and build Avatar QEMU

cd <halucinator_root>/deps/avatar2/targets/src
git clone AVATAR_QEMU_REPO
cd ..  (<halucinator_root>/deps/avatar2/targets)
./build_qemu.sh

Install Halucinator Make sure you are in you virtual environment and then run
```
pip install -r src/requirements.txt
pip install -e src
```
in the root HALucinator directory

Set environmental variables

export HALUCINATOR_QEMU_ARM=<HALUCINATOR_ROOT>/deps/avatar2/targets/build/qemu/arm-softmmu/qemu-system-arm
export HALUCINATOR_QEMU_ARM64=<HALUCINATOR_ROOT>/deps/avatar2/targets/build/qemu/aarch64-softmmu/qemu-system-aarch64
export HALUCINATOR_QEMU_M68K=<HALUCINATOR_ROOT>/deps/avatar2/targets/build/qemu/m68k-softmmu/qemu-system-m68k

Simlink gdb-multiarch to arm-none-eabi-gdb If you don't have arm-none-eabi-gdb on your path symlink gdb-multiarch to it. It which was installed in step 1. Just symlink it to arm-none-eabi-gdb
```
sudo ln /usr/bin/gdb-multiarch /usr/bin/arm-none-eabi-gdb
```

Note on setting HALUCINATOR_QEMU

If you use virtualenvwrapper as above you can set it up to be automatically set and removed when activating/deactivating the virtual environment using the postactivate and predeactivate scripts below.

Contents of $VIRTUAL_ENV/bin/postactivate

  export HALUCINATOR_QEMU_ARM=<HALUCINATOR_ROOT>/deps/avatar2/targets/build/qemu/arm-softmmu/qemu-system-arm
  export HALUCINATOR_QEMU_ARM64=<HALUCINATOR_ROOT>/deps/avatar2/targets/build/qemu/aarch64-softmmu/qemu-system-aarch64
  export HALUCINATOR_QEMU_M68K=<HALUCINATOR_ROOT>/deps/avatar2/targets/build/qemu/m68k-softmmu/qemu-system-m68k

Contents of $VIRTUAL_ENV/bin/predeactivate

unset HALUCINATOR_QEMU_ARM
unset HALUCINATOR_QEMU_ARM64
unset HALUCINATOR_QEMU_M68K

Running

Running Halucinator requires a configuration file that lists the functions to intercept and the handler to be called on that interception. I usually split this config across three files for portability. The files are a memory file that describes the memory layout, an intercept file that describes what to intercept and a symbol/address file that maps addresses to symbol names. See the Config File section below for full details

All of these commands assume you are in your halucinator virtual environment

halucinator  -c=<memory_file.yaml> -c=<intercept_file.yaml> -c=<address_file.yaml>

Running an Example

Building STM MX Cube Examples

This has already been done for Uart example file below.

A tool to convert the STM's Software Workbench for STM (SW4STM) was developed to enable compiling their IDE projects using make. This has only been tested on a few STM32F4 examples from STM32Cube_F4_V1.21.0. It compiles them as cortex-m3 devices and not cortex-m4 to enable easier emulation in QEMU.

To use go into the directory below the SW4STM32 directory in the project and run python3 <halucinator_repo_root>/src/tools/stm_tools/build_scripts/CubeMX2Makefile.py . Enter a name for the board, and the applications. Then run make all. The binary created will be in bin directory

Example

cd STM32Cube_FW_F4_V1.21.0/Projects/STM32469I_EVAL/Examples/UART/UART_HyperTerminal_IT/SW4STM32/STM32469I_EVAL
python3 <halucinator_repo_root>/src/tools/stm_tools/build_scripts/CubeMX2Makefile.py .
Board: STM32469I_Eval
APP: Uart_IT
make all

STM32F469I Uart Example

To give an idea how to use Halucinator an example is provided in test/STM32/example.

Setup

Note: This was done prior and the files are in the repo in test/STM/example. If you just want to run the example without building it just go to Running UART Example below.

This procedure should be followed for other binaries. In list below after the colon (:) denotes the file/cmd .

Compile binary as above
Copy binary to a dir of you choice and cd to it: test/STM32/example
Create binary file: <halucinator_repo_root>/src/tools/make_bin.sh Uart_Hyperterminal_IT_O0.elf creates Uart_Hyperterminal_IT_O0.elf.bin
Create Memory Layout (specifies memory map of chip): Uart_Hyperterminal_IT_O0_memory.yaml
Create Address File (maps function names to address): Uart_Hyperterminal_IT_O0_addrs.yaml
Create Intercept File (defines functions to intercept and what handler to use for it): Uart_Hyperterminal_IT_O0_config.yaml
(Optional) create shell script to run it: run.sh

Note: Symbols used in the address file can be created from an elf file with symbols using hal_make_addrs This requires installing angr in halucinator's virtual environment. This was used to create Uart_Hyperterminal_IT_O0_addrs.yaml

To use it the first time you would. Install angr (e.g. pip install angr from the halucinator virtual environment)

hal_make_addrs -b <path to elf file>

Running UART Example

Start the UART Peripheral device, this a script that will subscribe to the Uart on the peripheral server and enable interacting with it.

hal_dev_uart -i=1073811456

In separate terminal start halucinator with the firmware.

workon halucinator
<halucinator_repo_root>$./halucinator -c=test/STM32/example/Uart_Hyperterminal_IT_O0_config.yaml \
  -c=test/STM32/example/Uart_Hyperterminal_IT_O0_addrs.yaml \
  -c=test/STM32/example/Uart_Hyperterminal_IT_O0_memory.yaml --log_blocks -n Uart_Example

or
<halucinator_repo_root>& test/STM32/example/run.sh

Note the --log_blocks and -n are optional.

You will eventually see in both terminals messages containing

 ****UART-Hyperterminal communication based on IT ****
 Enter 10 characters using keyboard :

Enter 10 Characters in the first terminal running hal_dev_uart press enter should then see text echoed followed by.

 Example Finished

Stopping

Press ctrl-c. If for some reason this doesn't work kill it with ctrl-z and kill %, or killall -9 halucinator

Logs are kept in the tmp/<value of -n option>. e.g tmp/Uart_Example/

Config file

How the emulation is performed is controlled by a yaml config file. It is passed in using the -c flag, which can be repeated with the config files being appended and the later files overwriting any collisions from previous file. The config is specified as follows. Default field values are in () and types are in <>

machine:   # Optional, describes qemu machine used in avatar entry optional
           # if never specified default settings as show in () below are used. 
  arch: (cortex-m3)<str>,
  cpu_model: (cortex-m3)<str>,
  entry_addr: (None)<int>,  # Initial value to pc reg. Obtained from 0x0000_0004
                        # of memory named init_mem if it exists else memory
                        # named flash
  init_sp: (None)<int>,     # Initial value for sp reg, Obtained from 0x0000_0000
                        # of memory named init_mem if it exists else memory
                        # named flash
  gdb_exe: ('arm-none-eabi-gdb')<path> # Path to gdb to use


memories:  #List of the memories to add to the machine
  - name: <str>,       # Required
    base_addr:  <int>, # Required
    size: <int>,       # Required
    perimissions: (rwx)<r--|rw-|r-x>, # Optional 
    file: filename<path>   # Optional Filename to populate memory with, use full path or
                      # path relative to this config file, blank memory used if not specified
    emulate: class<AvatarPeripheral subclass>    # Class to emulate memory 

peripherals:  # Optional, A list of memories, same as memories except emulate field required

intercepts:  # Optional, list of intercepts to place
  - class:  <BPHandler subclass>,  # Required use full import path
    function: <str>     # Required: Function name in @bp_handler([]) used to
                        #   determine class method used to handle this intercept
    symbol: (Value of function)<str>  # Optional, Symbol name use to determine 
                                      # address in firmware to intercept, name 
                                      # must be present in symbols,
                                      # If not use value of function is used
    addr: (from symbols)<int>  # Optional, Address of where to place this intercept,
                               # generally recommend not setting this value, but
                               # instead setting symbol and adding entry to
                               # symbols (in seperate file) as this makes config 
                               # files more portable. If set will take precidence over symbol
    class_args: ({})<dict>  # Optional dictionary of args to pass to class's
                       # __init__ method, keys are parameter names
    registration_args: ({})<dict>  # Optional: Arguments passed to register_handler
                                   # method when adding this method
    run_once: (false)<bool>   # Optional: Set to true if only want intercept to run once
    watchpoint: (false)<bool> # Optional: Set to true if this is a memory watch point

symbols:  # Optional, dictionary mapping addresses to symbol names, used to
          # determine addresses for symbol values in intercepts
  addr0<int>: symbol_name<str>
  addr1<int>: symbol1_name<str>

options: # Optional, Key:Value pairs you want accessible during emulation

Config file

How the emulation is performed is controlled by a yaml config file. It is passed in using a the -c flag, which can be repeated with the config file being appended and the later files overwriting any collisions from previous file. The config is specified as follows. Default field values are in () and types are in <>

machine:   # Optional, describes qemu machine used in avatar entry optional defaults in ()
           # if never specified default settings as below are used. 
  arch: (cortex-m3)<str>,
  cpu_model: (cortex-m3)<str>,
  entry_addr: (None)<int>,  # Initial value to pc reg. Obtained from 0x0000_0004
                        # of memory named init_mem if it exists else memory
                        # named flash
  init_sp: (None)<int>,     # Initial value for sp reg, Obtained from 0x0000_0000
                        # of memory named init_mem if it exists else memory
                        # named flash
  gdb_exe: ('arm-none-eabi-gdb')<path> # Path to gdb to use


memories:  #List of the memories to add to the machine
  - name: <str>,       # Required
    base_addr:  <int>, # Required
    size: <int>,       # Required
    perimissions: (rwx)<r--|rw-|r-x>, # Optional 
    file: filename<path>   # Optional Filename to populate memory with, use full path or
                      # path relative to this config file, blank memory used if not specified
    emulate: class<AvatarPeripheral subclass>    # Class to emulate memory 

peripherals:  # Optional, A list of memories, except emulate field required

intercepts:  # Optional, list of intercepts to places
  - class:  <BPHandler subclass>,  # Required use full import path
    function: <str>     # Required: Function name in @bp_handler([]) used to
                        #   determine class method used to handle this intercept
    addr: (from symbols)<int>  # Optional, Address of where to place this intercept,
                               # generally recommend not setting this value, but
                               # instead setting symbol and adding entry to
                               # symbols for this makes config files more portable
    symbol: (Value of function)<str>  # Optional, Symbol name use to determine address
    class_args: ({})<dict>  # Optional dictionary of args to pass to class's
                       # __init__ method, keys are parameter names
    registration_args: ({})<dict>  # Optional: Arguments passed to register_handler
                              # method when adding this method
    run_once: (false)<bool> # Optional: Set to true if only want intercept to run once
    watchpoint: (false)<bool> # Optional: Set to true if this is a memory watch point

symbols:  # Optional, dictionary mapping addresses to symbol names, used to
          # determine addresses for symbol values in intercepts
  addr0<int>: symbol_name<str>
  addr1<int>: symbol1_name<str>

options: # Optional, Key:Value pairs you want accessible during emulation

The symbols in the config can also be specified using one or more symbols files passed in using -s. This is a csv file each line defining a symbol as shown below

symbol_name<str>, start_addr<int>, last_addr<int>

alex-bellon/halucinator