yfeng95/DECA

why dose the loss of photo_detail_mrf is nan?

kingo233 opened this issue · 11 comments

Hi! thank you for your excellent work.
I have finished pretrain and corase shape train,and the model's ablibity to extract pose,light,tex is good.
But when I started trainning detail,I got photo_detail_mrf is nan .
How can I solve it?

@kingo233 Hello, I'm also trying to reproduce the training process. However, after the coarse training, I couldn't get the right code['light'] while the other latent codes are well learned, I wonder can you give me some suggestions?

@zydmu123 Hi! Please give me more infomation about this problem.Can you show me your training image ?

@kingo233 Thanks, I have fixed the light, just by switching the below codes:
# ops = self.render(verts, trans_verts, albedo, codedict['light']) ops = self.render(verts, trans_verts, albedo, h=h, w=w, background=background)
and my coarse training results for now are as follows:
00039350

However, a similar problem as yours seems to arise during my detail training...

@zydmu123 Hi! I haven't solved this problem. Can you give your contact infomation? Want to communicate with you more directly

@zydmu123 Hey,bro. Please check your email.Our team member have sent you an email talking about ours situation.

@kingo233 @zydmu123 Hello, I train coarse well. But detail result very bad. Have you trained detail well? My wechat: xuduo1029

orig_11067_inputs
2023-07-28 14-18-22 的屏幕截图
2023-08-04 17-29-00 的屏幕截图

zek1th commented

Hi! thank you for your excellent work. I have finished pretrain and corase shape train,and the model's ablibity to extract pose,light,tex is good. But when I started trainning detail,I got photo_detail_mrf is nan . How can I solve it?
我也遇到这个问题了,能问下你是怎么解决的吗?还有,文中说在训练细节模型时,粗模型是固定的,这代表我要将训练粗模型时的loss都置为0吗?还是需要什么别的操作啊。如能回复,感激不尽

我也遇到这个问题了,能问下你是怎么解决的吗?还有,文中说在训练细节模型时,粗模型是固定的,这代表我要将训练粗模型时的loss都置为0吗?还是需要什么别的操作啊。如能回复,感激不尽

这代表你的训练配置文件要把is_detail设置为true,训练时代码会走到另外一个分支,粗模型部分不会得到更新

zek1th commented

我也遇到这个问题了,能问下你是怎么解决的吗?还有,文中说在训练细节模型时,粗模型是固定的,这代表我要将训练粗模型时的loss都置为0吗?还是需要什么别的操作啊。如能回复,感激不尽

这代表你的训练配置文件要把is_detail设置为true,训练时代码会走到另外一个分支,粗模型部分不会得到更新

原来如此,感谢感谢,我已经设置cfg.train.train_detail = true。
但是另一个我跟你之前存在的问题一样,detail阶段的 photo_detail_mrf 值为 nan,这个你是怎么解决的呢?

原来如此,感谢感谢,我已经设置cfg.train.train_detail = true。 但是另一个我跟你之前存在的问题一样,detail阶段的 photo_detail_mrf 值为 nan,这个你是怎么解决的呢?

这个我也一直没解决,抱歉

zek1th commented

原来如此,感谢感谢,我已经设置cfg.train.train_detail = true。 但是另一个我跟你之前存在的问题一样,detail阶段的 photo_detail_mrf 值为 nan,这个你是怎么解决的呢?

这个我也一直没解决,抱歉

好吧,谢谢了:>