API updates to Blender and Tensorflow have rendered this project incompatible without significant tinkering to blender_project and its main dependency Lifting-from-the-Deep.
I do not have the time anymore to actively develop and support this project, but feel free to fork this project, poke around its inner workings and make your own adjustments. Pull requests are always welcome as well!
Kind regards, Haryo
blender_lifting is a Blender addon that implements the functionality from Denis Tome' research paper Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image, which uses machine learning to estimate a 3D pose from a 2D image. More information can be found about this can be found in its GitHub repository and its associated research paper.
To install this addon, use git clone https://github.com/Fragrag/blender_lifting.git to clone this repository to Blender's addon folder or download a .zip version of this repository and unpack it in Blender's addon folder. You should now have a blender_lifting folder in your addon folder.
Run the setup_windows.sh bash script. This will create the Python virtual environment blender_lifting_venv and install the required libraries, with exception of the lifting module.
The lifting module comes included in a folder in the root of this repository. After creating the virtual environment, simply copy or move this folder to the virtual environment's module library. In Windows this is will be in blender_lifting_venv\Lib\site-packages
Lifting From The Deep for Blender should now be installed properly.
Once installed properly, you can activate the addon in Blender in Edit->Preferences->Add-Ons as Animation: Blender Lifting.
The interface is then accessed when in Object Mode, under Tools.
- Image path: Location of the image to estimate 3D pose from
- Scale: Scale of armature
- Armature name: Name of armature
- Create an armature from a 2D image: Press here to create the estimated 3D pose
- ADVANCED: Probability Model: Location of the probability model used by lifting
- ADVANCED: Saved Sessions: Location of the sessions used by lifting
Software:
- Blender 2.80
- Python 3.7.3
Python libraries:
- SciPy
- TensorFlow
- OpenCV Python
- Scikit Image
- Lifting
- and the dependencies of the libraries above
Armature is not normalised The created armature is not normalised and will not be facing in a particular default direction. As a result the base bone might be positioned oddly as well.
Non standard armature The created armature does not conform to any standard rigs or skeletons that I am aware of. Consider the created armature as a starting point rather than a production ready asset.
My armature came out rather weird/inaccurate blender_lifting relies on a machine learning model to estimate a 3D pose from a 2D image. As such it might return an anatomically incorrect armature. The estimation is also not 100% accurate, and is more of an approximation.
It failed!/It crashed! blender_lifting is still in its early days of development and can be very buggy. Implementing a robust error catching is part of the future roadplan. In the meantime, make sure your image contains a single person with most, if not all, limbs clearly visible.
Also make sure that your Python 3.7.3 installation is added to your PATH variables as python.
Why do I need Python installed on my system when Blender has its Python installation While developing, Blender's built-in Python exhibited issues when running lifting's dependencies such as OpenCV and TensorFlow. A workaround was created by having the add-on launch a system-level Python process to analyse the image rather than the built-in Python.
This addon uses a lightly modified version of Denis Tome's Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image.