About the annotations of keypoints

Question

About the annotations of keypoints

Opened this issue 4 years ago · 19 comments

Dear @silviazuffi ,
Thanks for your sharing. Excellent works!
Could you please upload the code to generate keypoints of the input image?
I want to apply this to my customed dataset without keypoints. Could you please give me some suggestions of the
keypoints generation?

Thanks very much.

Answer 1 · 2020-09-19T06:34:31.000Z

Hi, We do not use key points on the input image, we only use keypoints during training, and these are obtained from the 3D avatars. You can just use the project_points function of the renderer to project selected 3D vertexes of the model on the image plane.

…

On 19. Sep 2020, at 03:54, sdf0615 ***@***.***> wrote: Dear @silviazuffi <https://github.com/silviazuffi> , Thanks for your sharing. Excellent works! Could you please upload the code to generate keypoints of the input image? I want to apply this to my customed dataset without keypoints. Could you please give me some suggestions of the keypoints generation? Thanks very much. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNYVFP3KDDMTSCKLXWCZUTSGQFLZANCNFSM4RSXBTHA>.

Answer 2 · 2020-09-19T06:43:55.000Z

Dear @silviazuffi ,
Thanks for your quick reply. I will try the method you told me.
Besides, I have another question. Given a full image, how could I process it as the cropped one. What operations did you do
when you process the original image. I guess you handle these images with horse detection and segmentation, as well as align all the images. Could please upload the process code?
Thanks very much,
yours
Wenfeng

Answer 3 · 2020-09-19T21:24:43.000Z

We got the images from Wildbook (https://zebra.wildbook.org/ <https://zebra.wildbook.org/>) through our co-author, and those were already as bounding boxes. I think you can run something like Mask R-CNN to identify the zebras

…

On 19. Sep 2020, at 08:44, sdf0615 ***@***.***> wrote: Dear @silviazuffi <https://github.com/silviazuffi> , Thanks for your quick reply. I will try the method you told me. Besides, I have another question. Given a full image, how could I process it as the cropped one. What operations did you do when you process the original image. I guess you handle these images with horse detection and segmentation, as well as align all the images. Could please upload the process code? Thanks very much, yours Wenfeng — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNYVFOJJSZ22IYSLCZXZQTSGRHLPANCNFSM4RSXBTHA>.

Answer 4 · 2020-09-20T07:16:49.000Z

Yes, definitely I could get the boxes and mask via Mask R-CNN, however, I saw your zebra images are all aligned so that the images are in the same locations and even all the heads of the zebra were in the right side, I wonder how to process all the images to get such results. In addition, whether this processing results will have an influence on the mesh model generation.

Answer 5 · 2020-09-20T07:27:14.000Z

We mirror the images, so they are not all from the same side.

…

On 20. Sep 2020, at 09:17, sdf0615 ***@***.***> wrote: Yes, definitely I could get the boxes and mask via Mask R-CNN, however, I saw your zebra images are all aligned so that the images are in the same locations and even all the heads of the zebra are in the right side, I wonder how to process all the images to get such results. In addition, whether this processing results will have a influence on the mesh model generation. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNYVFMTD5N6AP2NENKOK4TSGWT6ZANCNFSM4RSXBTHA>.

Answer 6 · 2020-09-20T07:29:13.000Z

Yes, I get it. Thanks for your kind reply.

Answer 7 · 2020-09-21T09:08:51.000Z

Dear @silviazuffi ,
I have sent you the results of my customer dataset, please check your emaill.silvia@mi.imati.cnr.it. Thanks very much,

Answer 8 · 2020-09-21T09:30:38.000Z

Are those images of Grevy’s zebras?

…

On 21. Sep 2020, at 11:09, sdf0615 ***@***.***> wrote: Dear @silviazuffi <https://github.com/silviazuffi> , I have sent you the results of my customer dataset, please check your ***@***.*** ***@***.***>. Thanks very much, — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNYVFMFYQZKB23XJZQRWODSG4J3HANCNFSM4RSXBTHA>.

Answer 9 · 2020-09-21T09:46:39.000Z

No， I apply it to some web searched horse images.

Answer 10 · 2020-09-21T09:56:03.000Z

The pre-trained network is trained on Grevy’s Zebras, it cannot be used for horses.

…

On 21. Sep 2020, at 11:46, sdf0615 ***@***.***> wrote: No， I apply it to some web searched horse images. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNYVFME4MT5UUNOFQWZCM3SG4OI5ANCNFSM4RSXBTHA>.

Answer 11 · 2020-09-21T13:08:02.000Z

Hi @silviazuffi ,
What should I do to train a new model for horses?

Answer 12 · 2020-09-21T13:24:14.000Z

You would need to create a synthetic dataset for horses like the one for the zebras. For doing this you need to have SMAL models of horses with variable textures and a 3D prior for sampling horse poses.

…

On 21. Sep 2020, at 15:08, sdf0615 ***@***.***> wrote: Hi @silviazuffi <https://github.com/silviazuffi> , What should I do to train a new model for horses? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNYVFILRWHGGBKV5QZ5ZSLSG5F4DANCNFSM4RSXBTHA>.

Answer 13 · 2020-09-25T06:34:01.000Z

hi @silviazuffi ,
I have read your works '3D Menagerie: Modeling the 3D Shape and Pose of Animals', This is great work.
I saw some demos of reconstructing 3d mesh from the 2d images. Could you send me this part of the code to do further research?

Answer 14 · 2020-11-05T07:29:54.000Z

hi @silviazuffi ,
I want to make myself SMAL model such as pig, but I don't know the model .pkl file. Now I have a .obj model which contains the information of v, vt, f...Could you please tell me how to make the .pkl model which contains J,betas?

Answer 15 · 2020-11-05T08:25:46.000Z

Hi, You should align the SMAL model to your 3D model. If you look at this repository https://github.com/silviazuffi/smalr_online <https://github.com/silviazuffi/smalr_online> there are functions to compute mesh distances. You can use those to perform the alignment. Best

…

On 5. Nov 2020, at 08:30, Autumnfall-Me ***@***.***> wrote: hi @silviazuffi <https://github.com/silviazuffi> , I want to make myself SMAL model such as pig, but I don't know the model .pkl file. Now I have a .obj model which contains the information of v, vt, f...Could you please tell me how to make the .pkl model which contains J,betas? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNYVFL2H7FPZO3XXZRPOJTSOJIABANCNFSM4RSXBTHA>.

Answer 16 · 2020-11-05T08:37:56.000Z

Dear @silviazuffi ,
Thanks for solving my confusion. I will try the method you told me.

Answer 17 · 2024-06-27T09:03:14.000Z

Dear Silivia,
I read the Supplemental Material for the SMALST model, where you mentioned generating a synthetic training set and training a network for horses . Is the dataset published anywhere? If not, can you share it with me? I'm interested in using it and comparing it to my current dataset Thank you in advance?

Answer 18 · 2024-06-27T11:08:34.000Z

Yes, I still have the dataset, it is about 9K images, I am quite busy in these days, but I will try to make it available.

Answer 19 · 2024-06-27T13:25:16.000Z

Thank you for your reply. I truly appreciate your willingness to make the dataset available

…

On Thu, 27 Jun 2024 at 13:08, silviazuffi ***@***.***> wrote: Yes, I still have the dataset, it is about 9K images, I am quite busy in these days, but I will try to make it available. — Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3RBGH2RDP5I6UTK6LR7HYTZJPXETAVCNFSM6AAAAABJ7PJHXWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJUGQYDGMBTHE> . You are receiving this because you commented.Message ID: ***@***.***>