Locks causing slow response times on POST /job
gwoo opened this issue · 3 comments
gwoo commented
I created a jobdb with a few thousand entries.
for i in {1..2000}
do
time curl localhost:8000/api/v1/job/ -d '{"command": "echo cool", "name": "test_job", "schedule": "R/2017-04-011T11:11:11-07:00/PT10S}'
done
Then continue to add more jobs in smaller increments. From time to time, a request would take >2s to complete.
I tracked the issue down to Persist holding the lock for an extended time. So, I started looking for a way to remove the locks. To solve the problem, I implemented a FreeJobCache that uses a thread safe, lock free hashmap: gwoo@a458d4c
You can also see, I modified the rest of the source to use the FreeJobCache. All the tests passed, so this seemed like a good change.
Probably the best thing to do would be to modify MemoryJobCache with the code from FreeJobCache. I can work on a PR if that you are interested.