How to test PTF/IPNet on one single case
hengtaoguo opened this issue · 7 comments
Hello! Thank you so much for this awesome work!
I am very new to this field, and currently we are trying to automatically fit an SMPL model to human surface point clouds (or mesh). I assume that the fit_SMPLD_PTFs.py should be very help to us. My question is that, given one case from the CAPE dataset (CAPE_ROOT/00145 for example), how should we run your scripts to get some fitting results? I am not sure how to set the subject_id or the sequence_id in this scenario.
Thank you again for your time and look forward to hearing from you!
Hi,
Thanks for your interest in our work!
The subject_idx is the integer index for the desired subject; it starts from 0 and it is based on the subjects specified in "test_split" entry of the yaml config file. For example, if you specify test_split = ['00122', '00145', '00159'] and you want to choose subject 00145, you should set subject_idx=1.
The sequence_idx is the integer index for the desired sequence; it also starts from 0. Sequences of each subject are sorted (
Line 139 in a8789c9
Hope this helps.
Hi, thank you so much for the response! It helps me get it working.
Using the following command, now I can generate meshes for a single case results, which is pretty amazing! I can visually see the 24 joints in different color:
python generate.py --subject-idx 1 --sequence-idx 0 configs/cape/ptf_decoder-width-256-128_ptfs-width-256-128_full-smpl_ce-ce_w-skin-1e-1_conv-encoder_hidden-256_plane64x3_softmax_npts-5000_CAPE-release-with-aug-trans_1gpus.yaml
What bothers me was that when I try to fit SMPLD to the generated meshes, the results are very weird. They can be seen from the Google Drive link I posted above. I was running the following command:
python smpl_registration/fit_SMPLD_PTFs.py --num-joints 24 --use-parts --init-pose configs/cape/ptf_decoder-width-256-128_ptfs-width-256-128_full-smpl_ce-ce_w-skin-1e-1_conv-encoder_hidden-256_plane64x3_softmax_npts-5000_CAPE-release-with-aug-trans_1gpus.yaml
Do you happen to have any ideas what may go wrong on my side? Thank you again for your patience!
Best
@Tonight1121 similiar problem here, any solutions now?
Hi, thank you so much for the response! It helps me get it working.
Using the following command, now I can generate meshes for a single case results, which is pretty amazing! I can visually see the 24 joints in different color:
python generate.py --subject-idx 1 --sequence-idx 0 configs/cape/ptf_decoder-width-256-128_ptfs-width-256-128_full-smpl_ce-ce_w-skin-1e-1_conv-encoder_hidden-256_plane64x3_softmax_npts-5000_CAPE-release-with-aug-trans_1gpus.yamlWhat bothers me was that when I try to fit SMPLD to the generated meshes, the results are very weird. They can be seen from the Google Drive link I posted above. I was running the following command:
python smpl_registration/fit_SMPLD_PTFs.py --num-joints 24 --use-parts --init-pose configs/cape/ptf_decoder-width-256-128_ptfs-width-256-128_full-smpl_ce-ce_w-skin-1e-1_conv-encoder_hidden-256_plane64x3_softmax_npts-5000_CAPE-release-with-aug-trans_1gpus.yaml
Do you happen to have any ideas what may go wrong on my side? Thank you again for your patience!Best
I was unable to reproduce the issue, can you specify how you changed the code? Besides can you also post the registration log file (under "generation" folder).
It seems most likely due to scale difference of generated mesh and the SMPL template. This shouldn't happen because the scale is already handled in the code, unless you changed it.. can you load both minimal.registered and minimal.posed and see how they align?
Hi thanks for the reply! I uploaded two more zip files containing the meshes and registration I generated so far in here. I did not change the codes in "fit_SMPLD_PTFs.py", except that I assigned subject-idx=1 and sequence-idx=0 as default in the parser. Here is my log file:
2021-07-18 20:13:24,519 Inner distance for input longlong_ATUsquat.000001: 0.3674798309803009 cm
2021-07-18 20:13:24,519 Outer distance for input longlong_ATUsquat.000001: 0.6497257351875305 cm
2021-07-18 20:13:24,525 Inner distance for input longlong_ATUsquat.000006: 0.364563912153244 cm
2021-07-18 20:13:24,526 Outer distance for input longlong_ATUsquat.000006: 0.6454629898071289 cm
2021-07-18 20:13:24,529 Inner distance for input longlong_ATUsquat.000011: 0.36974582076072693 cm
2021-07-18 20:13:24,530 Outer distance for input longlong_ATUsquat.000011: 0.6599771976470947 cm
2021-07-18 20:13:24,533 Inner distance for input longlong_ATUsquat.000016: 0.3722936809062958 cm
2021-07-18 20:13:24,534 Outer distance for input longlong_ATUsquat.000016: 0.6649101376533508 cm
2021-07-18 20:13:24,538 Inner distance for input longlong_ATUsquat.000021: 0.3721478581428528 cm
2021-07-18 20:13:24,538 Outer distance for input longlong_ATUsquat.000021: 0.6529878973960876 cm
2021-07-18 20:13:24,542 Inner distance for input longlong_ATUsquat.000026: 0.37511295080184937 cm
2021-07-18 20:13:24,542 Outer distance for input longlong_ATUsquat.000026: 0.677676260471344 cm
2021-07-18 20:13:24,546 Inner distance for input longlong_ATUsquat.000031: 0.3762461841106415 cm
2021-07-18 20:13:24,547 Outer distance for input longlong_ATUsquat.000031: 0.6848275661468506 cm
2021-07-18 20:13:24,551 Inner distance for input longlong_ATUsquat.000036: 0.374575138092041 cm
2021-07-18 20:13:24,551 Outer distance for input longlong_ATUsquat.000036: 0.6889687776565552 cm
2021-07-18 20:13:24,558 Inner distance for input longlong_ATUsquat.000041: 0.3744427561759949 cm
2021-07-18 20:13:24,558 Outer distance for input longlong_ATUsquat.000041: 0.6660463809967041 cm
2021-07-18 20:13:24,564 Inner distance for input longlong_ATUsquat.000046: 0.37237757444381714 cm
2021-07-18 20:13:24,564 Outer distance for input longlong_ATUsquat.000046: 0.6801172494888306 cm
2021-07-18 20:13:24,568 Inner distance for input longlong_ATUsquat.000051: 0.3739171326160431 cm
2021-07-18 20:13:24,568 Outer distance for input longlong_ATUsquat.000051: 0.6532168984413147 cm
2021-07-18 20:13:24,572 Inner distance for input longlong_ATUsquat.000056: 0.36724644899368286 cm
2021-07-18 20:13:24,572 Outer distance for input longlong_ATUsquat.000056: 0.6556229591369629 cm
2021-07-18 20:17:23,110 Inner distance for input longlong_ATUsquat.000061: 0.3729289472103119 cm
2021-07-18 20:17:23,111 Outer distance for input longlong_ATUsquat.000061: 0.6751689314842224 cm
2021-07-18 20:17:23,116 Inner distance for input longlong_ATUsquat.000066: 0.37142157554626465 cm
2021-07-18 20:17:23,116 Outer distance for input longlong_ATUsquat.000066: 0.6654493808746338 cm
2021-07-18 20:17:23,120 Inner distance for input longlong_ATUsquat.000071: 0.37089803814888 cm
2021-07-18 20:17:23,120 Outer distance for input longlong_ATUsquat.000071: 0.6663898825645447 cm
2021-07-18 20:17:23,124 Inner distance for input longlong_ATUsquat.000076: 0.3654461205005646 cm
2021-07-18 20:17:23,124 Outer distance for input longlong_ATUsquat.000076: 0.66495281457901 cm
2021-07-18 20:17:23,127 Inner distance for input longlong_ATUsquat.000081: 0.3606269061565399 cm
2021-07-18 20:17:23,128 Outer distance for input longlong_ATUsquat.000081: 0.6463714241981506 cm
2021-07-18 20:17:23,133 Inner distance for input longlong_ATUsquat.000086: 0.34528377652168274 cm
2021-07-18 20:17:23,133 Outer distance for input longlong_ATUsquat.000086: 0.6281700134277344 cm
2021-07-18 20:17:23,137 Inner distance for input longlong_ATUsquat.000091: 0.33335137367248535 cm
2021-07-18 20:17:23,137 Outer distance for input longlong_ATUsquat.000091: 0.5830997824668884 cm
2021-07-18 20:17:23,141 Inner distance for input longlong_ATUsquat.000096: 0.3192345201969147 cm
2021-07-18 20:17:23,141 Outer distance for input longlong_ATUsquat.000096: 0.5597406625747681 cm
2021-07-18 20:17:23,145 Inner distance for input longlong_ATUsquat.000101: 0.309969037771225 cm
2021-07-18 20:17:23,145 Outer distance for input longlong_ATUsquat.000101: 0.5552759170532227 cm
2021-07-18 20:17:23,150 Inner distance for input longlong_ATUsquat.000106: 0.3078795373439789 cm
2021-07-18 20:17:23,150 Outer distance for input longlong_ATUsquat.000106: 0.536182701587677 cm
2021-07-18 20:17:23,153 Inner distance for input longlong_ATUsquat.000111: 0.3086300492286682 cm
2021-07-18 20:17:23,153 Outer distance for input longlong_ATUsquat.000111: 0.5437053442001343 cm
2021-07-18 20:17:23,157 Inner distance for input longlong_ATUsquat.000116: 0.3103150725364685 cm
2021-07-18 20:17:23,157 Outer distance for input longlong_ATUsquat.000116: 0.5362607836723328 cm
2021-07-18 20:19:45,807 Inner distance for input longlong_ATUsquat.000121: 0.3095962405204773 cm
2021-07-18 20:19:45,807 Outer distance for input longlong_ATUsquat.000121: 0.5313876867294312 cm
2021-07-18 20:19:45,812 Inner distance for input longlong_ATUsquat.000126: 0.3072328567504883 cm
2021-07-18 20:19:45,812 Outer distance for input longlong_ATUsquat.000126: 0.5393871068954468 cm
2021-07-18 20:19:45,816 Inner distance for input longlong_ATUsquat.000131: 0.31105896830558777 cm
2021-07-18 20:19:45,816 Outer distance for input longlong_ATUsquat.000131: 0.5374715924263 cm
2021-07-18 20:19:45,819 Inner distance for input longlong_ATUsquat.000136: 0.3101412057876587 cm
2021-07-18 20:19:45,819 Outer distance for input longlong_ATUsquat.000136: 0.5422878861427307 cm
2021-07-18 20:19:45,823 Inner distance for input longlong_ATUsquat.000141: 0.3130332827568054 cm
2021-07-18 20:19:45,823 Outer distance for input longlong_ATUsquat.000141: 0.5313974618911743 cm
2021-07-18 20:19:45,827 Inner distance for input longlong_ATUsquat.000146: 0.31135979294776917 cm
2021-07-18 20:19:45,827 Outer distance for input longlong_ATUsquat.000146: 0.5247372984886169 cm
2021-07-18 20:19:45,831 Inner distance for input longlong_ATUsquat.000151: 0.3089272081851959 cm
2021-07-18 20:19:45,831 Outer distance for input longlong_ATUsquat.000151: 0.5195490717887878 cm
2021-07-18 20:19:45,835 Inner distance for input longlong_ATUsquat.000156: 0.3044670820236206 cm
2021-07-18 20:19:45,835 Outer distance for input longlong_ATUsquat.000156: 0.5201680064201355 cm
2021-07-18 20:19:45,839 Inner distance for input longlong_ATUsquat.000161: 0.29972586035728455 cm
2021-07-18 20:19:45,839 Outer distance for input longlong_ATUsquat.000161: 0.5103909969329834 cm
2021-07-18 20:19:45,866 Mean inner distance: 0.3427780568599701 cm
2021-07-18 20:19:45,866 Mean outer distance: 0.6059722900390625 cm
Hi,
Judging by the alignment of implicit meshes and SMPL meshes, I suspect something went wrong with the semantic labels of the SMPL mesh.
Can you try:
- Remove the "--use-parts" option when calling fit_SMPLD_PTFs.py, see if the registration gets better.
- Did you follow the "Build the Dataset" steps, especially step 0? Note that the code is based on SMPL v1.0 not v1.1. If possible, can you pack your "body_models" directory and share it with me? If something goes wrong with registration it's most likely due to these 3rd party weights...
Best
Closing the issue due to inactivity. Feel free to reopen it if your problem persists.