Deepstream6_YoloV5_Kafka
This repository gives a detailed explanation on making custom trained deepstream-Yolo models predict and send message over kafka.
-
Install NVIDIA deepstream-6.0/6.0.1/6.1
- Test if deepstream-test1 is working fine.
-
Keep your custom trained YoloV5 model yolov5s.pt ready
-
Clone Deepstream-Yolo (https://github.com/marcoslucianops/DeepStream-Yolo) repository at /opt/nvidia/deepstream/deepstream/sources/ location.
-
Clone yolov5 (https://github.com/ultralytics/yolov5.git) repository at any location say /home/ozsports/.
-
cd yolov5 -
Build a conda environment with python
-
pip3 install -r requirements.txt -
Move the custom trained Yolov5 model yolov5s.pt to yolov5 folder at location /home/ozsports/yolov5/.
-
Sudo copy the gen_wts_yoloV5.py file from /opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/utils directory to the /home/ozsports/yolov5/ folder.
-
Generate the cfg and wts files
python3 gen_wts_yoloV5.py -w yolov5s.pt -
Sudo copy the generated cfg and wts files to the DeepStream-Yolo folder location /opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/.
-
-
Open the DeepStream-Yolo folder and compile the lib
-
For DeepStream 6.1 on x86 platform
CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo -
For DeepStream 6.0.1/6.0 on x86 platform
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
-
-
Edit the /opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/config_infer_primary_yoloV5.txt file according to your model (example for YOLOv5s)
[property]
...
custom-network-config=yolov5s.cfg
model-file=yolov5s.wts
...
-
Edit the /opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/deepstream_app_config.txt file
...
[primary-gie]
...
config-file=config_infer_primary_yoloV5.txt
-
Copy labels.txt for the custom trained models and place it in /opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/ folder.
-
Test the model
deepstream-app -c deepstream_app_config.txt
-
If it works fine, move ahead
-
Open another terminal at /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-test5/
-
Open /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-test5/deepstream_test5_app_main.c file
- Follow IMPORTANT Note 1 and IMPORTANT Note 2 and make small edits in the file. File attached for reference.
-
Edit file /opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/deepstream_app_config.txt. File attached for reference.
- Change the input video file path
[source0]
...
#uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
uri=file:///home/ozer/Downloads/Main_1.mp4
...
- Introduce the sink for kafka message streaming
[sink1]
...
type=6
msg-conv-config=/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-test5/configs/dstest5_msgconv_sample_config.txt
msg-conv-payload-type=0
msg-broker-proto-lib=/opt/nvidia/deepstream/deepstream/lib/libnvds_kafka_proto.so
#msg-broker-conn-str=YOUR-IP;PORT;topic
msg-broker-conn-str=192.168.1.40;9092;test
#topic=topic
topic=test
- Edit the path of the config-file.
[primary-gie]
...
config-file=/opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/config_infer_primary_yoloV5.txt
-
Edit file /opt/nvidia/deepstream/deepstream-6.1/sources/DeepStream-Yolo/config_infer_primary_yoloV5.txt. File attached for reference.
- Preferably put absolute path whenever needed. Also edit num-detected-classes to your custom classes.
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
custom-network
config=/opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/yolov5s.cfg
model-file=/opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/yolov5s.wts
model-engine-file=/opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/model_b1_gpu0_fp32.engine
#int8-calib-file=calib.table
labelfile-path=/opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/labels.txt
batch-size=1
network-mode=0
num-detected-classes=4
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=/opt/nvidia/deepstream/deepstream/sources/DeepStream-Yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
-
Make sure you have kafka and kafka-composer in your machine.
-
Follow this (https://forums.developer.nvidia.com/t/using-kafka-protocol-for-retrieving-data-from-a-deepstream-pipeline/67626/13) post to set up kafka consumer and producer and test their working. It creates docker-compose.yml, producer.py and consumer.py. Make sure you have put your IP address in all the 3 files as well.
-
Open one terminal where you have docker-compose.yml. File attached for reference.
sudo docker-compose up -
Make sure kafka and zookeeper are running
sudo docker ps -
Open another terminal and in deepstream conda environment run consumer.py.
python3 consumer.py. File attached for reference. -
Make sure you have edited the /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-test5/Makefile
-
for Deepstream-6.0/6.0.1, CUDA_VER=11.4
-
for Deepstream-6.1, CUDA_VER=11.6
-
-
Open another terminal at /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-test5/ folder and make the files
-
sudo make clean -
sudo make all
-
-
Finally, run the deepstream-test5 app
./deepstream-test5-app -c ../../../DeepStream-Yolo/deepstream_app_config.txt -
You should see an On Screen Display with predictions and metadata on terminal with consumer.py running in a conda deepstream environment.
NOTE: Feel free to use deepstream.yml for conda environment installation. File attached.
