ezai_env

Windows version not tested starting version 1.4.4

Description

Docker container and conda virtual environment creation for doing AI on local machines, Kubernetes or SageMaker.

Pre-requisites for building

Linux: Python is Installed
Windows: Python is installed, preferably miniconda.

Usage

EZAI in docker environment with conda:

Note: This version maps $home directory and /mnt directory on the host to docker container, so be careful with modifying anything in these folders inside the container, it will also be modified on the host

Do the following first:

Download the two files: curl -O "https://raw.githubusercontent.com/armando-fandango/ezai_env/main/{.env,ezai-docker}"
modify .env file as needed, specially the itag
execute source ezai-docker.sh
now use any of the following commands

To run EZAI container

ezai_runc
- pulls the image if not present
- starts the container if not started
- enters the container
I generally create an alias in bash to run the notebooks from the container quickly: alias eznb='conda activate ezai && jupyter notebook --ip=* --no-browser'

To pull EZAI image:

ezai_pull_image

To remove EZAI container

ezai_rmc

To remove EZAI image

ezai_rmi

To build EZAI image - clone the entire repo first

ezai_build_image

EZAI conda environment on your local machine (no docker):

modify ezai-conda-req.txt as needed
modify ezai-pip-req.txt as needed
for linux: execute source ezai-conda.sh,ezai_conda_create --venv <location-of-env> --py-ver <python-version>
for windows: execute ezai-conda-create.ps1 -venv <location-of-env> -python-ver <python-version>
- <location-of-env>: Default is /opt/conda/envs/ezai on linux and C:/Miniconda3/envs/ezai on windows
- <python-version>: Defalult is 3.7.3
- You can supply your own requirements.txt files with --piptxt and --condatxt.
activate the environment with conda activate <location-of-env>
test the tensorflow and pytroch GPU with pytest -p no:warnings -vv

EZAI conda environment in AWS SageMaker instance:

The SageMaker instance should be already created and stopped before following these steps.

On your laptop

Configure awscli if not already done: aws configure
Clone the repo : git clone https://github.com/armando-fandango/ezai_env.git
Go to the ezai_env folder where you cloned the repo
source the file source ezai-conda
Set the SageMaker env: set_sagemaker_env -i <instance_name>

Prepare the SageMaker instance

Launch the instance and open a terminal
In the terminal: cd /home/ec2-user/SageMaker/ezai_env
Create conda environment: source ezai-conda.sh,ezai_conda_create --venv /home/ec2-user/SageMaker/envs/ezai
activate the environment with conda activate /home/ec2-user/SageMaker/envs/ezai
test the tensorflow and pytorch GPU with pytest -p no:warnings -vv

Test it

Open the test.ipynb file

TODO:

Give option to create fully isolated container
Give option to select which libraries to install
add/enable more libraries
make separate dockerfile for Sumo and create only if asked from the run

armando-fandango/ezai_env