TSID Performance Issue, slow HQP solves
Closed this issue · 9 comments
Hi,
I tried running Exercise 2 and noticed that the robot was moving very slow compared to the demo in the first 10 seconds of this video from the lecture Optimization-Based Robot Control - Lesson 4: https://www.youtube.com/watch?v=FJJ_y-dgIEc&list=PL4nPbSX5VFGg5Izt8hjyQ_2R8rdnn6qWg&index=5.
I added some print statements to see what was taking so long in each loop iteration and also saw what my CPU performance was:
- Generally,
- Time spent in compute sample 0.00003,
- Time spent in HQP setup 0.00020
- Time spent in HQP solve 0.026
- Time spent in one loop 0.027
The HQP solve time was between 0.023 and 0.027 seconds. And the total time spent in one loop was about 0.027 seconds, when it should have been 0.002 s (i.e., conf.dt).
- I observed that one of my CPUs was always at 100% during the program run, and it would sometimes switch to another core but one of them would always be at 100%.
Below you can see the total execution time to be about 107 seconds; in another test, it was about 120 seconds. The time should have been around 8 seconds, based on the number of samples, N, and sample rate, dt.
Any suggestions or explanations of this performance would be helpful. I am hoping to use TSID for the whole-body control block of my project but will not be able to use it if I can't achieve high frequency loops (0.5+ kHz).
Dear @hushmandesmaeili thanks for your interest in this software.
Could you give us more details on how you installed TSID ?
Dear @olivier-stasse, thank you for your prompt response.
I installed it using the instructions from the README:
cd $DEVEL/openrobots/src/
git clone --recursive git@github.com:stack-of-tasks/tsid.git
cd tsid
mkdir _build-RELEASE
cd _build-RELEASE
cmake .. -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=$DEVEL/openrobots
make install
To install the Python bindings, I:
sudo apt-get install robotpkg-py3*-eigenpy
cd ~/path/to/tsid/_build-RELEASE
cmake .. -DCMAKE_BUILD_TYPE=RELEASE -DBUILD_PYTHON_INTERFACE=ON
make
sudo make install
Gepetto is in ../site-packages and TSID is in ../dist-packages for some reason, so exporting both:
export PYTHONPATH=/opt/openrobots/lib/python3.10/site-packages:/opt/openrobots/lib/python3.10/dist-packages:$PYTHONPATH
A peer also ran it on their computer but with no issues. It took less than 10 seconds and CPU usage went up to about 8-10% on their computer. I think that the cause of the issue, then, is probably my computer hardware, which can be seen with my CPU usage going up to 100%; it might indicate that the CPU is the bottleneck. FYI, my CPU specification is: Intel(R) Xeon(R) CPU E3-1225 v5 @ 3.30GHz.
to use site-packages instead of dist-packages, you can use the -DPYTHON_STANDARD_LAYOUT=ON cmake option. A workaround of making a simlink between those two directories is also possible.
How did you install the dependencies of TSID ? If hpp-fcl is missing the Release mode, it would explain the bad timings.
@hushmandesmaeili we used TSID at 1KHz on a TALOS robot with 30 actuators and Intel i7 Core @ 2.8 GHz. So the first thing to check is indeed to see if any of the dependencies is missing the Release mode. This usually increases the computation time often by a factor of 20 (on our code).
@nim65s @olivier-stasse Thank you both for your responses.
To install eigen3:
sudo apt-get install libeigen3-dev
To install pinocchio, I used robotpkg. hpp-fcl was also installed as a depedency of Pinocchio:
sudo apt install -qqy robotpkg-py3*-pinocchio
To install eiquadprog:
git clone https://github.com/stack-of-tasks/eiquadprog.git
cd eiquadprog
mkdir build && cd build
cmake ..
make
sudo make install
To compile TSID and install the Python bindings:
cd ~/path/to/tsid/_build-RELEASE
cmake .. -DCMAKE_BUILD_TYPE=RELEASE -DBUILD_PYTHON_INTERFACE=ON
make
sudo make install
Other than TSID, which I explicitly set the build flag to Release mode, I'm unsure how to check if the other dependencies are missing the Release mode. Any insights on this would be greatly appreciated, thanks!
Thanks for the details.
I guess the next think to try would be to compare the performances with what you get on your machine with sudo apt install robotpkg-py3*-tsid, if you can check that.
@nim65s @olivier-stasse Thank you both for your responses.
To install eigen3:
sudo apt-get install libeigen3-dev
To install pinocchio, I used robotpkg. hpp-fcl was also installed as a depedency of Pinocchio:
sudo apt install -qqy robotpkg-py3*-pinocchio
To install eiquadprog:
git clone https://github.com/stack-of-tasks/eiquadprog.git
cd eiquadprog
mkdir build && cd build
cmake ..
make
sudo make install
It seems to me you haven't compiled eiquadprog in release mode. Instead of doing cmake .., you should do cmake .. -DCMAKE_BUILD_TYPE=RELEASE
@andreadelprete good catch ! This is probably the missing point.
@andreadelprete This worked! Great catch!
The time spent in one loop is 0.002 s as expected and the whole thing runs in less than 10 seconds. Thank you all for your help!