Could you provide some hints on how to achieve the performance of your pre-trained model from scratch?
windpls opened this issue · 16 comments
Hi trigeorgis, thanks for your nice code. I have compiled and run the training process but found there is a big gap between the model trained myself and the one you provided. Data augmentation is one factor as you have mentioned in README. Is there any other reason for this gap? And could you provide some details about your data augmentation? Thanks.
Same result as yours. I cannot train the model as the auther provided.
Thanks @windpls for letting me know. Could you please give some more detail on the results you currently obtain, along with the CED curve you obtain? I am to polish up the code soon, but it should still obtain comparable results as those in the paper.
@trigeorgis I trained the model very long time ago to set as a baseline. If I remember correctly, the error in the training is about 20%. Therefore, I did not even test the model.
The code has beed changed since then so I would suggest to try it again. I am also not sure what you mean by 20%. Do you mean that the total RMSE error was .2?
I mean NMRSE=0.2. I will wait new version of your code that support latest TF and try again. Now I do not have access of the machine that has old version bazel and cannot rebuild yours Tensorflow. :(
I ran your pretrained model with help of the provided ipython script and with this I achieved 41.14 AUC @0.08 on 300W test. I also trained a model however with that I only achieve 35.76 AUC as compared to the 45.32 AUC you report in the paper.
@ShownX Hello, I could not compile the extract_patches.cc in the tensorflow r0.11. You said slice operation and tf.stack could work. Could please give me more details about it? I have spent two days on this problem, but I have not solved it...Thanks!
@MoExplorer You can slice the tensor and join them together.
@ShownX Thanks for your reply. Do you mean that you have written another code that can slice the tensor and join them together to replace extract_patches.cc? I'm newbie in tensorflow. So if it's convenient, could you please provide the corresponding code as a reference for me ? I'll really appreciate it !
@MoExplorer I am happy to help you. But I cannot distribute any code without permission.
TensorFlow really make life harder. If you still want to, this page would definitely help.
@ShownX Thanks for your kindly help. I have another question here: Dose the tensorflow you have used to run the MDM code come from the source provided by the aurthor? Mine(r.0.8.0) is installed inside the anaconda, but I failed to compile the extract_patches.cc in it, which really confuses me now...
@MoExplorer I once compiled it successfully.
@superphil0 Hello, you said you have trained the model sucessfully and got another result. But when I train the model, the training process always stops halfway because of error "Model diverged with loss = NaN". I really don't know what's wrong... Have you ever met this error? If so, please help me, I'll really appreciate it!
No sorry, I didn't get this error, but you can try a lower learning rate? Good luck!
@superphil0 Thanks for your reply, and I will have a try. And I have some other questions here:
Is the max_step you set in the training model 100,000?
Whether the total training loss keeps unchanged after a certain step? I found that every time I restart to train the model, the unchanged value of training loss is different. Even sometimes it comes to zero, whcih really confuses me now... Could you please give me some advice? Thanks a lot!
Please have a look on the implementation provided in the experimental branch.
(It goes without saying that this is experimental)