are there any transformer models available on LightningPose App platform for training pose estimation models?

[themattinthehatt]

          @themattinthehatt , I am wondering if you could let me know, if any transformer models available on LightningPose App platform for training pose estimation models?

Originally posted by @ReetKaur15 in #49 (comment)

[themattinthehatt]

@ReetKaur15 we have some pre-trained SAM backbones you can access; see the test here for an example of how to use:
https://github.com/danbider/lightning-pose/blob/main/tests/models/test_heatmap_tracker_mhcrnn.py#L52

(basically you just need to set cfg.model.backbone to vit_b_sam)

We did some simple tests of this model back over the summer and didn't really find any improvements over resnet-50, but there are also a lot things we didn't try. Please let me know if you get something working better than conv nets!