GPU Version
Closed this issue · 5 comments
I don't suppose you've created a GPU version of this?
It looks like MediaPipe only supports GPU computing and rendering on Android, Linux, and iOS. On Android, it's always enabled. On Linux, you can enable it according to their docs by replacing the option --define MEDIAPIPE_DISABLE_GPU=1 with --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 in the bazel build command in build-x86_64-linux.sh. If you build libmediapipe now, the computations should run on the GPU, but I haven't tested this yet.
I was finally able to create a GPU version by creating another target in c/BUILD that includes the parts/pieces for GPU. That created the hand_landmark_gpu.binarypb and libmediapipe_gpu.so files that I could use and it worked.. sort of. It turns out that it ran about 4X slower than the CPU version. I still have not figured out why that is, but am still working on it.
Thank you for writing this library. The work you did on this API simplified the bigger mediapipe library in such a way that it is actually easy to understand now.
Glad to hear that this library has been useful to you!
I have looked into this some more and it looks like running calculators on the GPU is not as easy as I initially thought. In their GPU demo, they initialize GPU resources on the graph and use GPUBuffers instead of ImageFrames for the input images. They also submit their packets in an OpenGL context and call some OpenGL functions like glFlush directly. I'll make these changes to mediapipe.cpp and see if I can get it to run faster on my machine.
I have adapted the build script to compile MediaPipe with GPU support and added a new function mp_create_packet_gpu_image that creates a GpuBuffer packet from an mp_image. Creating and processing a GPU buffer packet is done in the OpenGL context. mp_create_instance now calls graph.SetGpuResources(*mediapipe::GpuResources::Create()) to initialize the GPU. I have also updated the example so that it uses the hand_landmark_tracking_gpu.binarypb graph and processes GPU image packets.
Strangely, the example runs at the same speed on my GPU as on the CPU, so I might still be doing something wrong? Or maybe MediaPipe simply isn't as optimized for desktop GPUs as it is for mobile GPUs...
Thanks for doing that. I was able to do something very similar in my local version but I saw much better speeds on GPU vs CPU when running on a ubuntu 22.04 machine. I originally was trying to run on wsl/ubuntu and that actually ran slower than the cpu version. Once I swap to a machine that had ubuntu installed on it I was able to get the full GPU speeds.