Standalone Test

Step 1: Run Development Docker

docker run -it --network=host \
  -v [your/path/to/clipper-develop]:/clipper \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /tmp:/tmp zsxhku/clipperpy35dev

Step 2: Go to clipper_admin dir

cd /clipper/clipper_admin

Step 3: Start DAG deployment

Test case 1: simple dag (No Prediction)

python simple_dag.py

Test case 2: predict stock price

python stock.py

Step 4: See the dockers/logs

docker container ls 
docker container logs [CONTAINER_ID]

Step 5: Stock DAG input / request

docker run -it --network clipper_network zsxhku/grpcclient [IP_OF_THE_ENTRY_PROXY] 22223

You can see the /grpcclient/app/grpc_client.py and see the implementations and implement you own grpcclient docker

But remember you should run the grpcclient docker under clipper_network

Step 5: Stop containers

python stop_all.py

Imagequery App

Code organization:

bigball: directory imagequery_bigball
clipper on raft: directory imagequery_clipper
our system(with proxy): directory imagequery_w_proxy
pure dag(without proxy): directory imagequery_wo_proxy

1. bigball

You only need to worry about imagequery_bigball

Go to directory: cd imagequery_bigball
Build image: ./build.sh
Run container: ./run.sh

2. clipper

You only need to worry about: boat_image, imagequery_clipper, clipper-develop/clipper_admin/imagequery_concurrent_client.py.

2.1 imagequery image

Go to directory: cd imaquery_clipper/container
Build image: ./build.sh

2.2 boat image

Go to directory: cd boat_image
Build image: ./build_imagequery_boat_image.sh
Run container: ./run_imagequery_boat_container.sh
Inside the container: python3 deploy.py to start boat.

2.3 Run client

Method 1
Open another terminal, connect to the same server, and make queries.
Reference: https://github.com/jitaogithub/boat
Useful commands are also included in comments in run_imagequery_boat_container.sh.

Method 2 (Not yet working, something wrong with imagequery_concurrent_client.py)

Open another terminal, start container

docker run -it --network=host \
-v [your/path/to/clipper-develop]:/clipper \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp zsxhku/clipperpy35dev

Go to directory: cd clipper/clipper_admin

Run client

python imagequery_concurrent_client.py \ 
--worker 1 --ip 127.0.0.1 --port 808x --system clipper

3. Without Proxy

You only need to worry about: imagequery_wo_proxy, clipper-develop/clipper_admin/imagequery_concurrent_client.py.

cd clipper/applications
./build_imagequery_wo_proxy
./run_imageuquery_wo_proxy
Note: if you get an error saying there is no clipper_network, run docker network create clipper_network, then use docker network ls to see if it is created.
Get port
docker inspect c0 | grep "IPAddress"

Run client

cd clipper-develop/clipper_admin  
python3 imagequery_concurrent_client.py \  
--worker 1 --port 22222 --ip [ip] --system withoutproxy

Note: There still seems to be something wrong with reducing and forwarding. Some of the output does not make sense. Use docker logs to debug. However, the problem should not be with the system, not our app :).

4. Our system / With Proxy

You only need to worry about: imagequery_w_proxy, clipper-develop/clipper_admin/imagequery_concurrent_client.py, clipper-develop/clipper_admin/imagequery.py.

cd clipper/applications
./build_imageuquery_w_proxy.sh. You can use run_imagequery_w_proxy.sh for testing.
Start container

docker run -it --network=host \
-v [your/path/to/clipper-develop]:/clipper \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp zsxhku/clipperpy35dev

cd clipper/clipper_admin
python3 imagequery.py and wait for 25 seconds (this is the safe time for the model to be properly set up).

Run client

The ip is specified as in the picture.

python imagequery_concurrent_client.py \
--worker 1 --ip 172.19.0.15 --port 22223 --system oursystem

Auto Pilot App

File structure, code organisation and naming convention is exactly the same with the above mentioned image query app.

Carefully replace image query with auto pilot at every step should give you desired result.

Stock Prediction App

Carefully Replace image query with stock prediction at every step should give you desired result.

Just be patient if you are dealing with 12 containers.

Sentiment Analysis App

Carefully Replace image query with sentiment analysis at every step should give you desired result.

Please note that current grpcclient may not support batch prediction and without_proxy at the same time. Need to be fixed later. And note that if you want to run container3: sentiment analysis on a single machine with GPU, it will occupy all GPU configuration. Therefore, please avoid setting c3 as stateful in dag_formatted in clipper-develop, sentiment directory. And also need to change c3 to cpu setting instead of gpu setting if you want to run wo_proxy. Else you may get CUDA lib not found error.

To build the whole app, run: python sentiment.py under clipper-develop/clipper_admin/

Fatigue Detection App

Carefully Replace image query with fatigue_detection at every step should give you desired result.

Please note that if you want to run container3 with GPU on a single machine, it will be very slow because of GPU competition. And if you set c3 as stateful, it may run out of resources on a single machine with GPU. In this case, please try machine with more GPU or avoid setting it as stateful.

And note that all build file can be found in fatigue_w_proxy directory. To build them please first run: mv build_docker_FatigueDetection.sh .. mv build_docker_fatiguewo.sh ..

To build the whole app, run: python fatigue.py under clipper-develop/clipper_admin/

Dumpkin1996/clipper