(1) Incorrect location: Image/.txt (2) in_place.py not working
HIN0209 opened this issue · 26 comments
Hello,
This is a great project and I enjoy training now with my custom jpg images (Ubuntu14.04, GTX1080Ti). I report two minor errors and eventually run successfully.
1: When I did "make train", there was an error message like, 'there is no file in ./Images/1.txt'. Of course, there is no 1.txt in the Images folder, because these are created in the devkit/2017/Labels. I copy/pasted these .txt files to the Images folder. That was enough to run successfully. The script needs to change to look for the label (.txt) files in the Labels, not in the Images folder.
2: scripts/in_place.py did not work to me. I simply found the respective parts in the easy.cfg and easy.data manually.
3: Regarding the testing a static image, it was a bit harder to solve! I wanted a jpg image to test, but it was designated as png. I looked into scripts and changed as follows:
(a) in pythonWrapper.py
fp = "test.jpg"
#fp = "test.png"
(b) in Makefile at the end,
test: ${PWD}/darknet detector test cfg/easy.data cfg/easy.cfg ${PWD}/backup/easy_final.weights ${PWD}/test.jpg
#test:${PWD}/darknet detector test cfg/easy.data cfg/easy.cfg ${PWD}/backup/easy_final.weights ${PWD}/test.png
Then, 'make train' yielded a jpg image with a predicted bounding box. However, I was puzzled with long error massages like:
Cannot load image "data/labels/32_0.png"
Cannot load image "data/labels/33_0.png"
I have two questions on this error. First, I cannot find a 'data' folder. Where is it? Second, how can I fix to reflect my training on jpg images?
Thank you!!
To self-respond, the .txt files should be located in the Images folder in case of original darknet, so I guess that the .txt must be moved to the Images folder after they are created.
@HIN0209 Sorry for the confusion.
I am aware of some small bugs and will update it soon.
Hi, to followup. There seems to have two typos.
In "easy_label.py"
(1)
def convert_annotation(year, image_id):
in_file = open('devkit/%s/Annotations/%s.xml'%(year, image_id))
#out_file = open('devkit/%s/Labels/%s.txt'%(year, image_id), 'w') #incorrect
out_file = open('devkit/%s/Images/%s.txt'%(year, image_id), 'w') #correct
(2)
for year, image_set in sets:
if not os.path.exists('devkit/%s/Labels/'%(year)):
os.makedirs('devkit/%s/Labels/'%(year))
#image_ids = open('devkit/%s/ImageSets/Main/%s.txt'%(year, image_set)).read().strip().split() #incorrect
image_ids = open('devkit/%s/ImageSets/%s.txt'%(year, image_set)).read().strip().split() #correct
By correcting these, the following will be created (The images are to be in the devkit/2017/Images, while the xml files are in the devkit/2017/Annotations).
(1) train.txt and val.txt in ImageSets
(2) .txt (location of BBox) in Images to be coexistent with image (.jpg, etc).
Hello,
Thank you for inviting. But I am a novice Pythoner and it is much beyond my capability!!
Please help anyone @HIN0209 @KleinYuan
@nikhiljaiswal Any details?
This looks like your GPU does not have enough memories.
@KleinYuan I don't think that's the GPU error, i think there is some sort of confusion in the cfg/easy.data file. Please have a look.
Here are the details of my entire file structure :
Here are the details in 2017_train.txt
Here are the details of the devkit/2017/Images folder
@nikhiljaiswal
Sorry man,
I fixed all issues now and tested on my machine, it should run now!
The root cause is that the original code hard coded the JPEGImages as the folder to contain the images, and labels as the label folders. I modified the src code before but forgot to push them here.
Now, all pipeline should be ready to go.
Please pull the new commit and try it again. It works on my machine.
@nikhiljaiswal specifically, this commit. Now just pull the new master branch.
Also, some announcement here to make.
This repo is relatively old, and I modified some src code (not critical ones but those configuration ones) so that it looks more friendly.
However, just be aware that the darknet original code base is way ahead of me and the main purpose of this repo is just to help whoever interested in object detection task to get started from scratch, step by step from environment preparations and to ensure this entire pipeline is always working not depending on too much moving parts. which potentially may break something someday in the future.
If you are really specifically interested in Yolo and wanna catch up with the most updated codes, you probably wanna go to original darknet repo.
However, if you are just interested in object detection problems, for more sophisticated/scalable way, you can refer to one of my another repo
@KleinYuan the other repo which u are referring basically consists of tensorflow object detection api. I just want to ask that have you used that to train on our custom dataset. I tried once, but I was getting some error.
@nikhiljaiswal Tensorflow should be quite easy to train with and check this out if you are interested.
But all in all, this repo should also be fixed now. Let me know whether you can run your training or not. Tensorflow/model's datasets organization is a little bit different than classic VOC tho.
@KleinYuan yes the data format is different. I had used it once. I am trying to adjust the modification you have made in this repo to my code. will report you the result as soon as I have modified and tested it.
@KleinYuan Still the same error. I don't understand why in cfg/easy.data , u have written names=data/easy.data
There is no data folder available anywhere
@nikhiljaiswal
How do you organize the folder?
Ensure the organizations are same as this.
easy.data seems like a another bug that I forgot to sync with my local changes. But this shouldn't impact the training process.
@nikhiljaiswal ahhhh
damn... I think this confuse you.
What I mean is that, just put images into Images folder. dataSets are the folder you use for labelling.
I will correct that soon.
hmmm. are you in my slack channel? I can help you live debug.
@nikhiljaiswal then could you join the slack channel and let's debug together?
Cuz, I tried three times run this repo from scratch and I didn't have any issues on running the training. Weird...
@KleinYuan
I must have done something wrong in my implementation. I have not used slack before. Please give me half an hour time. I would be thankful to you, if you help me in my debugging.
@KleinYuan
I have joined your channel
Thanks for the suggestions. The model has started training. Hope it works correctly & give desired results on my dataset. closing the issue Close #1
To recap, the root cause is @nikhiljaiswal used OpenCV 3 instead of OpenCV 2, in case people in future meet same issue.
If your results are not ideal, you wanna check your datasets firstly.