stereolabs/zed-yolo

darkent

Docker300 opened this issue ยท 36 comments

darknet.sln issue

@Docker300 You can try to build this repository https://github.com/AlexeyAB/darknet by using Cmake, there will be created uselib.exe that uses ZED-camera:
uselib.exe data/coco.names cfg/yolov3.cfg yolov3.weights zed_camera

I am unable to find uselib.exe file in darknet folder when I compiled this repository according to the instructions given in How to compile on Windows (using vcpkg)

If you have already installed Visual Studio 2015/2017/2019, CUDA > 10.0, cuDNN > 7.0, OpenCV > 2.4, then compile Darknet by using C:\Program Files\CMake\bin\cmake-gui.exe as on this IMAGE: Configure -> Optional platform for generator (Set: x64) -> Finish -> Generate -> Open Project -> x64 & Release -> Build -> Build solution

I have Visual Studio 2017, CUDA 10.0, cuDNN 7.5, OpenCV 3.4.0, CMAKE 3.13.4

When I use uselib.exe data/coco.names cfg/yolov3.cfg yolov3.weights zed_camera
it gives following error

'uselib.exe' is not recognized as an internal or external command,
operable program or batch file.
Where is this file located?

If you compiled with x64 & Release https://hsto.org/webt/uh/fk/-e/uhfk-eb0q-hwd9hsxhrikbokd6u.jpeg
then uselib.exe will be created in the Release folder.

Thank You for the help
I am newbie. I compiled Darknet.sln with x64 & Release. and when I use

uselib.exe data/coco.names cfg/yolov3.cfg yolov3.weights zed_camera

in command prompt it give error

uselib.exe System error

pthreadVC2.dll does not exist so the code execution can not proceed. So reinstall the program can resolve this issue.

uselib system error

I don't know why is it giving this error because project is successfully build

buildsuccess

But pthread.h is also not found in CMAKE.

Looking for pthread.h
Looking for pthread.h - not found

Dnet

I also get the same error pthreadVC2.dll for darknet.exe
"darknet.exe detector demo cfg/coco.data cfg/yolov3.cfg yolov3.weights zed_camera"

dsk

Would you please guide me ?
As pthread.h is not found in CMAKE and if it is the reason of getting pthreadVC2.dll error for uselib.exe and darknet.exe, then how can I solve it?
Do I need to re-install another version of CMAKE? or Do I need to make any change in any file to solve this issue? or Do I need to specify the environment variable for pthread.h and path for pthreadVC2.dll?

  1. Copy darklib.dll and uselib.exe from darknet/Release folder to the darknet/ root folder.

  2. Also copy pthreadGC2.dll and pthreadVC2.dll from darknet\3rdparty\pthreads\bin\ folder to the darknet/ root folder.

  3. Download https://pjreddie.com/media/files/yolov3.weights to the darknet/ root folder.

  4. Then run uselib.exe data/coco.names cfg/yolov3.cfg yolov3.weights zed_camera
    in the darknet/ root folder.

Thank You for the detailed reply :)

When I run uselib.exe data/coco.names cfg/yolov3.cfg yolov3.weights zed_camera
I get this error

input image or video filename: ZED 3D Camera NOT ENOUGH GPU MEMORY
Error: ZED Camera should be connected to USB 3.0. And ZED_SDK should be installed.

But ZED camera is connected and ZED SDK is also installed in C:\Program Files (x86)\ZED SDK
(CMAKE has also detected ZED SDK at the same location C:\Program Files (x86)\ZED SDK)

I do not get this error when I run ZED SDK Example projects.
But I get this error when I run ZED-YOLO.

Would you please guide me that why am I getting : ZED 3D Camera NOT ENOUGH GPU MEMORY
error and how should I solve it?

I am using Nvidia GeForce 840M and latest version of ZED SDK 2.8

d1
.
.
.
d2

It seems that you don't have enough GPU-RAM to run full yolov3-model.

Try to run tiny model:

uselib.exe data/coco.names cfg/yolov3-tiny.cfg yolov3-tiny.weights zed_camera

Thank You. Although, It is not showing "Not enough GPU" error now.
But it gives another error "ZED 3D Camera Failed to setup" and stops when I run either Yolo3 or Yolo-Tiny

I don't know that why it is giving this error .
For testing the camera, I opened ZED explorer to check if camera is setup or not. In ZED explorer it opens the camera, it means there is no issue related to camera setup. Then why it is giving this error here when I run it for YOLO

Please guide me
Do I need to make some changes in any file?

image

Line# 340 in yolo_consol_dll.cpp is already commented.
I am using Nvidia GeForce 840M (GPU).

Un-comment, i.e. change this line:
//init_params.sdk_cuda_ctx = (CUcontext)detector.get_cuda_context();
to this
init_params.sdk_cuda_ctx = (CUcontext)detector.get_cuda_context();

I did it, saved it and run the command but facing the same issue :(

I am getting this error and tried to fix it many times.

ZED (Init) >> ERROR: Executable was designed for SDK v2.8.0 / installed v2.8.1
CAMERA FAILED TO SETUP

Although, It build the project successfully but does not show any output due to this error.
I am facing this problem not only in YOLO but also in all other ZED examples now. Problem is still there after reinstalling the SKD
Would you please guide me?

@Docker300 Try to recompile Darknet Yolo

I tried every possible solution except reinstalling windows...
After re-installing Zed-SDK I also re-installed cuda 10.0 ( after deleting its old installation)
I recomiled Darknet YOLO but getting same message
image

When I run C:\Program Files (x86)\ZED SDK\tools\ZED Depth Viewer.exe
It shows the camera.
But
I don't know that why it fails to start camera when I run the code either darknet-YOLO or Zed master examples.
I have been trying to solve this issue since last two weeks.
Please suggest me some solution.

ZED (Init) >> ERROR: Executable was designed for SDK v2.8.0 / installed v2.8.1
CAMERA FAILED TO SETUP

What is the current SDK version do you use?

It seems that you try to run old SDK (2.8.0) exe-files

So it seems that somewhere there are still old and new versions of ZED SDK

  1. Try to delete all files of old SDK

  2. Also try to delete your Darknet, download it again and compile again.

I did all these steps and I'm using SDK 2.8.1 now.
After deleting SDK 2.8.0 it does NOT give message

ZED (Init) >> ERROR: Executable was designed for SDK v2.8.0 / installed v2.8.1

But

CAMERA FAILED TO SETUP

problem is still there when I run ZED Examples.

Also, gives the same message message with I run YOLO after downloading and compiling it.
Although, it does not give any error in build
But I don't know that whey it fails to run camera in code or SDK
while camera is successfully detected by zed explorer.exe

Thank You, It is working now.
Would you please tell me that
To train ZED-YOLO 3D ( with depth estimation for 3D object detection)

  • Do I have to use labeled 3D image dataset or labeled 2D image dataset?
    Or
  • Labeled dataset of both 2D images and labeled dataset of its Depth images?

You should use 2D labels as usual. ZED will estimate 3D coords automatically.

Okay.
Thank You :)

Same command is used to train the yolo-tiny with depth estimation for ZED?

darknet.exe detector train data/obj.data yolov3-tiny.cfg darknet53.conv.74
Or
we have to use different command?

Please guide me
In YOLO-ZED
What is x and y?
Is it height and width of bounding box?
and How z is calculated?
Does ZED_YOLO extracts the distance or z (depth) value by taking the center of bounding box(width/2, height/2) of detected image?

The current ZED Yolo simply takes the 2D detector output and compute the 3D world coordinates from the depth map computing at runtime during the inference.

The training phase doesn't take into account the 3D at all. It's just a regular Yolo 2D detector

Okay. Thank You
Would you please explain about x, y and z value here?
x, y values indicate the width and height of bound box while z shows the distance from camera?

image

@Docker300 x, y, z - are 3D coordinates of center of object (measured in meters relative to left camera)

Okay. Thank You

For training

  1. Is it okay If image size is 450 X 600 (instead of 450 x 450 or 600 x 600) ?
  2. In this case, should we also change the width and height in .cfg file according to our image size?
  3. if Yes then should we change other parameters according to 450 x 600 image size?

I have trained YOLOv3on my own dataset. With generated weights, when I use Zed camera in live mode it just draws bounding boxes.
For example, on test image and video files, it does not show labels and depth info when I run uselibe.exe it only draws bounding boxes
But when I run darknet.exe it shows the bounding boxes with labels
What would be the issue that uselib.exe only shows bounding boxes without labels or depth information?

Because you use wrong names-file in your command:
uselib.exe data/coco.names cfg/yolov3.cfg yolov3.weights zed_camera

It can show depth info only on Video-stream from ZED-camera.

You are right. Thank You :)

I can't able to use wingpudll compiled by visual studio the file do exists at path and os.exists also returns true but CDLL is not able to open the dll picture is attached please check
ss

why it doesn't work
Screenshot (5)

Thank You, It is working now.
Would you please tell me that
To train ZED-YOLO 3D ( with depth estimation for 3D object detection)

  • Do I have to use labeled 3D image dataset or labeled 2D image dataset?
    Or
  • Labeled dataset of both 2D images and labeled dataset of its Depth images?

How did you get this to work? I am having the same error. But I am on a brand new PC with RTX2070, so I know I have enough RAM and the proper GPU.

I get pthreadvc2.dll not found. How I can fix it?

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment otherwise it will be automatically closed in 5 days