/ndif

The NDIF server, which performs deep inference and serves nnsight requests remotely

Primary LanguagePythonMIT LicenseMIT

NDIF Development Guide

This guide explains how to set up a development environment, install dependencies, and get started with contributing to the NDIF project.

Prerequisites

  • Python 3.10
  • Docker
  • Docker Compose

Setup

1. Install Conda

If you haven't already, install Conda by downloading and installing Anaconda or Miniconda from the official Conda website.

2. Create Conda Environment

Fork or clone the NDIF repository to your local machine. Then create a new Conda virtual environment:

conda create -n ndif-dev python=3.10
conda activate ndif-dev

3. Install NNsight

Choose one of the following methods:

a. Via pip (simple)

pip install nnsight

b. From repository (recommended for specific branches)

git clone https://github.com/nnsight/nnsight.git
cd nnsight
git checkout <branch-name>  # e.g., 0.3
pip install -e .

4. Configure Docker Compose

Modify the Docker Compose file for your system

  1. Edit the file:
vi compose/(dev|prod)/docker-compose.yml
  1. Update the Hugging Face cache path:
- /disk/u/[YOUR BAULAB USERNAME]/.cache/huggingface/hub/:/root/.cache/huggingface/hub
  1. Adjust GPU settings (check your system with nvidia-smi):
- driver: nvidia
 count: 1
 capabilities: [ gpu ]

Building and Running NDIF

  1. Build the base environment

First, build the base environment using the make build_all_base command. This will set up the base images.

make build_all_base
  1. Build the service

Next, build the service using the make build_all_service command.

make build_all_service
  1. Start the development containers

After building the base environment and the service, start the NDIF docker containers.

make up-dev
  1. Verify server status

After building the NDIF containers, you can check the docker logs to verify the services are running correctly.

docker logs dev-api-1

You should expect to see a message like Application startup complete. in the api service log.

  1. Run tests
python scripts/test.py

This will send 3 NNsight requests to the API service running in the local container.