pose_transformer_v2 (BERT style transformer) use while running model on video

Question

pose_transformer_v2 (BERT style transformer) use while running model on video

Opened this issue 2 months ago · 3 comments

In the run tracking demo on videos, the paper mentions that the BERT-style transformer model(pose_transformer_v2) enables future predictions and amodal completion of missing detections within the same framework.

However, in the PHALP.py script, after running the (pose_transformer_v2), its output is deleted at line 260(in PHALP.py), and I can't find the model output/values used anywhere.

Where exactly does the code utilize the pose transformer v2? Is it involved in the rendering process?

Answer 1 · 2024-05-13T23:30:37.000Z

The pose transformer is used to predict future poses for each tracklet (and compare them with the detected poses when doing identity tracking). These future poses are not visualized. Currently, we only visualize the single-frame estimates from HMR2.0.

Answer 2 · 2024-05-15T16:01:55.000Z

Thanks for clarifying!

Answer 3 · 2024-06-14T15:08:03.000Z

Has there been any work done to evaluate the performance of this pose_transformer_v2(BERT style transformer)?
I have looked into the LART paper but the transformer model looks different from 4D-Humans.