svd did not converge
Magallan1229 opened this issue · 3 comments
Hi,
These days I have two new questions. The first one is that when I use softimp as me method , it always raise "svd did not converge" when preparing data but I dont understand what happened and how to solve
it. The second one is when I use nucnorm as me method, the acc rate of MNIST evaluation under fgsm0.3 is only nearly 20% when training after 20 epochs ,however ,usvt can acheive more than 80% after the same epochs. Is it a bug or I just should wait for training more epochs?
somebody says "svd did not converge" is caused by a matrix for svd that has nan or inf but I don`t find any in my matrix .
Finally I choose to use python keyword "try" to avoid the raise, which makes few (less than 10) images not be preprocessed. I think it wont cause a large drop of acc. After the full 200 epochs , acc of MNIST is only around 40% under fgsm0.3(softimp p=0.3 as me method). If I use usvt as me method, that acc is around 87%,which is consistent with fig6. The only thing I changed is me-type so I really dont know why that happened, do you have any idea?
Hi, sorry for the late reply. For MNIST, we did not test much with different ME methods, as the improvements are limited. We hypothesis that since there are almost only black/white pixels, the information on the image is not as rich as other datasets, e.g., CIFAR. The optimization based methods can be numerically unstable under this case.
I would suggest you try different parameters of other ME types, and could also visualize how the recovered images look like for the optimization methods. This might help debug whether the hyper-parameters are reasonable. Note that the current code settings are mainly extensively tested for CIFAR.