TF-Injector

A Versatile Fault Injector for ML Applications

TF-Injector is a software fault injector for TensorFlow v2 applications written in Python 3. A variety of faults are supported at the data, hardware and software levels.

1. Dependencies

TensorFlow framework (v2.0 or greater)
Python (v3 or greater)
PyYaml (v3 or greater)
numpy package (part of TensorFlow)

2. Installation and runs

Following are the installation and usage instructions for a Linux platform.

Clone the repository.

git clone https://github.com/nniranjhana/TF-Injector.git

Set the python path for the TF-Injector project so that it can be executed from other scripts. You can also add it permanently to .bashrc if you prefer.
```
export PYTHONPATH=$PYTHONPATH:$TF-INJECTORHOMEPATH
```
where $TF-INJECTORHOMEPATH might be like /home/nj/TF-Injector
You can navigate to conf/ to check out how to set the fault injection configuration for the tests you plan to run.
Let's see an example of how to mutate the weights of a model. Go to tests/tfv2/model and set the sample.yaml file in tests/tfv2/model/confFiles. If you are running from the examples in this directory, this is the file that gets picked up.

Run the test to observe the fault injection. For example, let's say we run the simple neural network example:

python nn-mnist.py

with the following configuration:

Artifact: 0
Type: mutate
Amount: 1
Bit: 30

This means that a single bit is flipped (30th bit position) in the model parameter tensor that holds the first hidden layer weights.

The results for this run are:

nj@nj-arch-xps:~/TF-Injector/tests/tfv2/model$ python nn-mnist.py

WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.

2020-05-06 17:02:42.180771: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version

step: 100, loss: 2.033674, accuracy: 0.578125
step: 200, loss: 1.827772, accuracy: 0.738281
step: 300, loss: 1.724117, accuracy: 0.808594
step: 400, loss: 1.675569, accuracy: 0.843750
step: 500, loss: 1.646916, accuracy: 0.859375
Test accuracy before injections: 0.829200

Test accuracy after injections: 0.734800