MATRIX

MATRIX is an MPC Automation Framework developed by Bar Ilan University Cryptography Research Group. It automates the tedious process of deploying, running, monitoring and summarizing results. It uses AWS or Azure to provision servers(instances), and can be used internally on a local host or in a container deployment.
A paper featuring MATRIX was prensented in ACM-CCS18

The system requires a management computer (Manager) - a computer that centralized all the execution. The Manager executes all the experiment phases, starting from install the experiment up to analyse it's results. The Manager is a stand alone workstation and it's not active member at the protocol.

In order to use all MATRIX capabilities, a cloud account is required.
MATRIX uses two cloud providers(CP):

AWS
Azure

To create account at AWS:

Sign up for AWS.
Define your credentials at the Manager computer.

To create account at Azure:

Installation

MATRIX runs under python 3.6 and uses fabric, fabric3 and openpyxl.
Matrix tested on these OSs:

Ubuntu 16.04.3/18.04.1 LTS
CentOS 7.3
Arch Linux

To install Python 3 and pip under Ubuntu 16.04/18.04 :

sudo apt-get install python3 python3-pip

To install under CentOS 7.3:

sudo yum install python35u.x86_64 python35u-pip.noarch

To install under Arch Linux:

pacman -S python python-pip

After You installed Python 3 and pip3 you will need to install the modules MATRIX uses. To install this modules use pip3

pip3 install --user -r requirements.txt

NOTE[1]: on some computers the following error may appear: locale.Error: unsupported locale setting To fix it, run:

sudo apt-get clean && sudo apt-get update && sudo apt-get install -y locales
locale-gen en_US.UTF-8

NOTE[2]: If you want to deploy your experiment at Azure, you will need to install the Azure-CLI.
To install it, run: curl -L https://aka.ms/InstallAzureCli | bash

After the modules installed, clone this repository to install MATRIX on your system.

MATRIX Modules

Deployment

After config file was created, You will need to deploy your images(instances). MATRIX supports three different deployments:

Local deployment
Servers deployment
AWS deployment

Local Deployment

To deploy MATRIX locally in CloudProviders let the name of the provider be local.
An example of a local deployment configuration can be found in here

AWS Deployment

After you created your AWS account and set your credentials (NOTE: MATRIX will fail if credentials are not set, see how above), you will need a key to deploy your instances. Detailed explanation can be found here. After you created your key, change this line at fabfile.py:

set the correct location and name of your AWS key env.key_filename = ['YOUR KEY HERE']

Azure Deployment

To set your credentials at Azure run az login and follow the instructions.

Execution

The execution module supports these operations:

Pre process - Operations that need to be done before the protocol executed like installation of library.
Install - Install the experiment.
Update - Update the current experiment if change was done to his code.
Execute - execute the protocol.
Results - Collect the results file from the images and analyse them. For more details see ExperimentReport section.
analyse - analyse the results from given directory.

NOTE: If you are using MATRIX on localhost, create your parties.conf file by using option 4 at the Deploy menu before start executing the protocol.

Reporting

MATRIX analyse four parameters:

CPU runtime (milliseconds)
RAM usage (GB) - Will be added in future release
Sent bytes (bytes) - Will be added in future release
Received bytes (bytes) - Will be added in future release

The report module analyse the results files that was taken from the images by number of parties parameter. If you want to analyse by different parameter use the Elasticsearch option.

MATRIX uses a header class logger API. The logging is done at the protocol code. The logger generate logs files that uploaded to Elasticsearch server. To use MATRIX logger class, just include MatrixMeasurement.h to your main class.

To measure task:

#include "MatrixMeasurement.h"
...
int main(int argc, char* argv[])
{
    MatrixMeasurement matrixLogger(argc,argv,vector<string>{"offline","online"}, numberOfIterations);
...
    matrixLogger.startSubTask("offline", iterationIdx);
    offline.run();
    matrixLogger.endSubTask("offline", iterationIdx);
...
}

MATRIX Configurations

GlobalConfigurations

In order to connect to the instances MATRIX uses a file that contains the AWS keys and security groups. For each region in AWS you need to create an entry in the global configuration file. Sample configuration file can be found here

ProtocolsConfigurations

NOTE: This section relevant only when using the CLI.

MATRIX uses configuration file to set it execution. The configuration file is written in json format. Each configuration file has the following fields:

protocol - Name of protocol
CloudProviders - for each cloud provider we need to create a unique entry. each entry contains these fields:
- numOfParties - number of instances to create.
- instanceType
- spotPrice - relevant only to AWS. For detailed explanation about spot instances, use this link
- git:
  - gitAddress - Git repository path. MATRIX will clone the repository into all target servers, configure, make and install.
    If installation of other libraries is needed to be done, see pre-process section of MATRIX for details.
  - gitBranch - The branch the protocol uses.
executableName - The name of the executable to execute
preProcessTask - ID of the pre process task that required. Not relevant to all of the protocols. The available pre process tasks that defines in MATRIX can be found in this script
Configurations - List of configurations to run. Each configuration is a set of CLI arguments to the executable. The arguments are separated between them by '@'. Party ID is added automatically
numOfRepetitions - How Many times MATRIX will execute the protocol
numOfInternalRepetitions - How many times the protocol will be executed on single run.
isPublished - Indicate if the protocol was published.
isExternal - Indicate if the protocol external to libscapi library.
regions - AWS regions to execute the protocol.
workingDirectory - The directory of the protocol and the data related to the protocol.
resultsDirectory - Directory to copy to the results files from the servers.The directory is local directory at the MATRIX system computer.
emails - MATRIX will send notifications to this email addresses. Multiple email addresses are supported
institute - Research Group identifier
coordinatorConfig - If coordinator exists in the protocol, the configuration for him will described here. The configuration need to be in the same format of 'configurations' field
coordinatorExecutable - The name of the coordinator executable

MATRIX Usage

After the installation you have two options to use MATRIX. CLI and Angular web UI.
There is no difference between the two of them in manner of functionality.

CLI

The CLI allows access to the MATRIX system. To run the CLI run: python3 main.py

Web UI

MATRIX UI developed under Angular 8 and enables simple UI. It uses Mongodb as for storing data about the protocol.
To install and deploy the web UI read the install instructions.

For bugs/features requests open an issue or send an email to liork.cryptobiu@gmail.com

Shunliing/MATRIX