Multi-Node webGPU support
Opened this issue · 1 comments
adminy commented
Cloudflare workers support webGPU, is there a way to take advantage of the fact that you get tons of worker nodes? I guess workerd also has to talk to the Vulkan / Metal API anyways but good thing that's still abstracted.
Btw Amazing work! It doesn't need benchmarks to tell its quite close to what the real hardware could give you.
cryscan commented
Thanks! I am currently working on a new version of the inference runtime which gets rid of waiting time in some CPU-bounded cases, and multi-device inferencing is on its roadmap!