Distributed computing issue when deployed on multiple nodes with kubernetes
Opened this issue · 1 comments
OmarSastec commented
I have deployed piper in kubernetes using 5 piper workers/pods .
After proceeding with the tests, I have noticed that all workers are responsible for a single job which is a problem in case I want to make some POST-TRANSCODING actions with the video.
Here's an explained scenario :
- worker X gets the job and started transcoding the file and saving the output to a temporary file
- worker X finished the transcoding
- worker Y tries to make some POST-TRANSCODING actions with the video => Here come the problem, file not found in the worker's tmp directory.
I did a hack using a mounted NFS volume to sync the workers processing but this is very Network IO consuming.
So my question: Is it possible to bind a unique worker to a unique job ?
cranky-coder commented
just curious if you have your deployment files. I am moving this direction and would like to be able to assist