/OpenVINO-Training

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

OpenVINO-Training

  1. Setup environment
    1. Windows
      1. Install Python (3.11.8)
      2. Setup Virtual Environment
      3. Install required libraries.
    2. Or you can try on Google Colab
  2. Run on Jupyter Lab
  3. Next steps...

Setup environment

On this training, please setup your environemt OpenVINO 2024.0 and Python 3.11.8 For running LLM models, I recommend to use Intel 12th Gen platform or later. If you don't have such system, please setup on Google Colab (instructions later).

System recommendation

Windows

  1. Install Python access to Python 3.11.8. Download Windwos 64 installer and click to open. Check "Add Python.exe to Path" and "Install Now"

  2. Setup Virtual Environment

    Open comandline prompt (cmd.exe). You can confirm python 3.11.8 is installed with typing "python". (exit() for finshing python prompt)

    cd %USERPROFILE%
python -m venv ov_env
ov_env\Scripts\activate

    then you can see as below. When finished type "deactivate"

  3. Download "requirements.txt" to under "ov_env" folder to install required libraries. Then type as below.

    cd %USERPROFILE%ov_env
pip install -r requirements.txt

On Google Colab

  1. Download "ov_training_llm_colab.ipynb"
  2. Go to Googl Colab. (you need google account yourself.)
  3. Click "Open Colab"
  4. "Upload" -> select "ov_training_llm_colab.ipynb"
  • Google Colab uses nVidia GPU which OpenVINO doesn't support, So device is set to CPU as inference engine.
  • if you failed at importing optimum.intel, please "restart session" and run process again.

Let's run LLM on Jupyer Lab

Let's dive into LLM local enablement process. ov_training_llm.ipynb

This time, we use Tiny Llama, very lightweight model for time restrction, but your can apply to almost LLM models on Hugging Face with same process :)

Next .. your turn!

Now you'll work over 100 samples on OpenVINO(TM) Notebooks!! https://github.com/openvinotoolkit/openvino_notebooks

You know how to setup environment on your system.

  • you may need to install git command for windows system
  • or youcan download zip file from the git link directly
python -m venv sampleenv
sampleenv\Scripts\activate
cd sampleenv
git clone https://github.com/openvinotoolkit/openvino_notebooks.git
cd openvino_notebooks-latest
pip install -r requirements.txt

then you can run samples in notebooks directory.

In note books, sample scripts are listed which can run on Jupyter Lab, try them!

First steps:

Here are some recommendataions you'll have fun with.