Heatmap gets blur after trained with the warped detector loss
triangleCZH opened this issue · 2 comments
Hi, thank you for your excellent work!
I am using this project to train for my personal satelite dataset, and I encounter the problem that, after I train the data with the warped detector loss, the model's inferences on the training data are being less precise, check the two heatmap 1_pretrain.jpg and 2_with_warp.jpg. The second one (after trained carefully) finds more points, but each point covers a larger area and the center is not as confident and bright.
I feel like the training is not sufficient, but not sure how to improve. I wonder if I could get some suggestions from you, and here is an introduction of how I use this repo.
How I train the model
I have to start the original model (SuperPointNet_pretrained.py in your project) and load its original weight (superpoint_v1.pth)
step2: I crop my high resolution images to 360x360 pieces as train/ val data, and export the label with the aforementioned model weights, based on 100 homographic aggregation.
step3: Data will keep (360, 360) during training, and I also load the pretrain as initial weight; I wanted to use the original dense loss, but it seems to be memory costly so I keep with the sparse loss. Then I train directly on step3, for 3000 iters (I use a small dataset to do experiment, only 400 images, bs=16) with lr=0.0001, then another 3000 iters with lr=0.00001.
Observations
When I export label for the second round, I find out those heatmap on a same image data are blur and not so bright (a grey area instead of just one shiny point)
Alternations
I try to disable the warped loss and the descriptor loss, to let it "overfit" on the its own exported label, and I do get an overfited result, the points are bright and normally just cover 1 pixel (as desired), check 3_without_warp.jpg
Please let me know if you have encountered anything similar, or could you please give me some suggestions about the training tricks? Thank you very much in advance!
1_pretrain.jpg
2_with_warp.jpg
3_without_warp.jpg
Additionally, I notice that we have a residua loss option for training, which seems to use the prediction after nms as input, do you think this is related to getting a more precise result? If not, as I did not see others discuss about it, may I know what purpose is this loss for? Thank you!
Getting blurry seems to be an expected behavior and does not affect results in a long run