About the tensorpack dataflow code
xiaofeiwu opened this issue · 5 comments
Hi Michal,
I used to work with the rmpt_dataset_server version of the code, and it works well. When I see that you have updated the code to tensorpack dataflow, which you say it could become faster, I become quite interested. The training becomes much easier.
However, as I can see that the code is not running really well. The loss stopped at 1200 after about two days. I think it may be because that it has used all the COCO data. Could you explain about the problem?
I use Python 3.6, tensorflow-gpu (1.8.0), and tensorpack (0.8.5) for the training. Thanks!
Xiaofei
Hi @xiaofeiwu Yes, I've noticed the problem. Unfortunately, my new rig is not yet ready so I am not able to do any training. I will fix that soon.
@michalfaber Thanks for the information. I'll be glad to see any update.
@michalfaber I tried to look into the problem of data generation in the current version. I found that a sample in generated for each person, not for each image. I wonder whether this could be the reason why the code is generating wrong result.
@zhihaolee were you able to fix the problem with your idea? I am stuck on training too, it is not learning properly.
This issue seems to be resolved in #103 @piperod @zhihaolee @xiaofeiwu