Prevent containers from reading jobs
mdavis332 opened this issue · 2 comments
Scenario: attempting to create a scalable, HA cluster, i'm using a backing MISP database with both a read/write instance and a read-only replica. My intent is to use a rev proxy/load balancer in front of environment to route API requests (like /events/restSearch or /attributes/restSearch) to the containers that are looking at the read replicas. I call these the API-only containers. Then programmatic queries to the instance won't affect the human analysts being routed to the "normal" MISP containers which point to the read/write instance of the MYSQL_HOST. Everything else gets routed to a normal container with a MYSQL_HOST pointing to the r/w db instance.
However, the issue I'm finding is that the containers pointed to the read replicas are still picking up jobs that have been initiated like fetch feed jobs. They fetch fine, but then try to save the "completed" state to the db replica which doesn't work (and gets an error about the SQL host being in read-only mode). I tried setting the REDIS_HOST envvar on those API containers to a fake host so they wouldn't pick up the jobs (thinking redis wouldn't be useful for those containers), but then the container just repeatedly restarts b/c it can't reach a valid redis host.
Additionally, I set all the JOBBER_ time string envvars to blank so no jobs would run on the API containers. However, the issue persists and the API-only containers still read jobs from redis, run the jobs, then fail to save the state.
Any idea how this can be resolved?
@mdavis332
As my pull request got merged, you are able to set the workers on the read-only host to zero. This will prevent the instance from executing jobs.
@mdavis332 As my pull request got merged, you are able to set the workers on the read-only host to zero. This will prevent the instance from executing jobs.
This did the trick, @Benni0. Thanks so much for the addition and tagging me on the merge!