Can I link with OpenPose?
Closed this issue · 5 comments
I would like to predict human behavior using 3D coordinates obtained with OpenPose, is it possible?
Our code assumes that the input is given in joint angles. Open pose produces 2d joint detection, so this sounds hard.
One way might be to
- Get Open pose detections,
- Then convert them to 3d using our other project, in particular this code.
- Then do inverse kinematics to recover the joint angles
And then you could use this code.
Unfortunately not easy 🙂
Thank you for your kind reply.
If I suppose that I guided the joint angle, how can I use the program you made to predict the behavior?
Please follow the README. You have to train a model and then take samples from it.
@taka0913 I have been doing something similar, but using OpenPose and a stereo camera to get 3D poses.
I've tried to use this code to do some forecasting but there are some additional considerations. eg. the Human 3.6M skeletons and the OpenPose (OP) output skeletons do not have the same number or location of joints.
You could think about doing some interpolation or the like to get the required to joints (eg. the tailbone is missing in the OP skeletons but you could take it to be the mid-point of the two hip joints, which are in OP). Of course, this is assuming you want to use one of the models trained on Human 3.6M. Otherwise, you could train your own on data obtained from OP.
I still haven't figured out how to give the networks custom inputs and just forward prop it. The easiest way seems to be to modify encoder_inputs, decoder_inputs, decoder outputs with new data and call a model.step(), but your data needs to be in the correct format, which is unclear to me.
@ajdroid @una-dinosauria have either of you written an IK solver in python for the H36M dataset to link this with OpenPose?