Geo-Cloud Workshop

Tutorial: How to run Madagascar FWI [1] on a Jupyter-Docker container [2].

1. Create a Google Compute Engine (GCE) instance.

First, we will grant access to the instance via Internet to use Jupyter notebooks.

For that, we need to enter (or create) the Google Cloud project and create a firewall rule:

1.1. Click to the Menu:

1.2. Go to Networking and open the Firewall rules option:

1.3. Type the name of the firewall rule (e.g. jupyter):

1.4. Type the tag (you can use the same as the rule name; e.g. jupyter), type the Source IP range 0.0.0.0 and type the tcp port 8888:

1.5 Verify the jupyter tag is OK:

1.6. In the Project Menu, go to Compute Engine and choose the VM instances option:

1.7. Type the name of your instance (e.g. gpu-team1). In Machine type, select Customize and add one GPU device:

1.8. Change the default Boot disk, select Ubuntu 16.04 TLS and add more capacity to your disk (e.g. 50 GB):

1.9. Allow the access to HTTP traffic, open the Management,security... options, type your tag (jupyter) in Networking and create your instance:

1.10. Once your instance is created, you can access to it by opening a terminal in a browser window (SSH protocol):

2. Install CUDA and verify Docker containers can access the GPU.

2.1. Access as root (you may use a simple password [e.g. 1234]):

sudo passwd
su

As root:

2.2. Clone the geo-cloud installation files (i.e. CUDA, docker and nvidia-docker):

git clone https://github.com/fernandovr89/geo-cloud-install.git

2.3. Install CUDA (Give execution access if needed chmod +x ./geo-cloud-install/install_cuda.sh):

sh ./geo-cloud-install/install_cuda.sh

2.4. Verify CUDA tools can access the GPU device:

nvidia-smi

2.5. Install Docker and nvidia-docker (give execution access if needed chmod +x ./geo-cloud-install/install_nvidia-docker.sh):

sh ./geo-cloud-install/install_nvidia-docker.sh

2.5. Verify Docker containers can access the GPU device:

docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi

3. Run the gpufwi script using Jupyter.

3.1. Run the dockerized Jupyter notebook [3] using the following command:

docker run --runtime=nvidia -it -p 80:8888 \
   fernandovr89/cuda:jpytr_ahay /opt/conda/bin/jupyter notebook \
   --notebook-dir=/opt/notebooks --no-browser --allow-root \
   --ip=0.0.0.0 --port=8888

Note: It will automatically download a (docker image)[https://hub.docker.com/r/fernandovr89/cuda] with Madagascar and CUDA installed.

3.2. Copy the token (the number after ...?token=):

3.3. In the GCP window, open Jupyter by clicking the external http IP of your instance:

3.4. Paste the token and log in:

3.5. Create a new Text File:

3.6. Copy the SConstruct file from this script, name it and save it (this is a script with the instructions to run FWI on GPU devices using Madagascar [4]):

3.7. Create a new Python 2 Notebook:

3.8. Copy the instruction cells from this notebook

Great! After a few minutes you can obtain your first Geo-Cloud results

Extras

If you want to continue working on the Cloud. You can run the script of the Reverse Time Migration (RTM) GPU implementation [5] on Marmousi model. scons script.

Note: You better create a new folder with a new SConstruct file using the Jupyter notebook.

IMPORTANT!

Don't forget to delete your instance. Otherwise, it will continue running and increasing the bill!

I.1. Select the instance and click on the delete button.

References:

[1] Madagascar Webpage - Seismic processing software

[2] Medium.com post of @allenday: Jupyter + Tensorflow + Nvidia GPU + Docker + Google Compute Engine

[3] Docker image of @fernandovr89: Madagascar - Seismic processing software running on Ubuntu 16.04TLS

[4] A graphics processing unit implementation of time-domain full-waveform inversion by Pengliang Yang, Jinghuai Gao, and Baoli Wang: Geophysics, 80, F31-F39, (2015) scons script

[5] RTM using effective boundary saving: A staggered grid GPU implementation by Pengliang Yang, Jinghuai Gao, and Baoli Wang: Computers & Geosciences, 68, 64-72, (2014) scons script