Recommended Settings for IO bound jobs?
Closed this issue · 10 comments
Hi there all! I was wondering if anybody have any recommended settings(as in GOMAXPROCS and number of workers) for IO bound jobs that I can start with? If you're wondering what IO bound jobs I'm doing, I'm accessing a remote mysql database. Would like some suggestions if you have any. Thanks!
"IO bound jobs" => goroutines
GOMAXPROCS => processes/containers goroutines can be scheduled on
Personally I always use GOMAXPROCS=NUMCPU+1. For I/O bound goroutines, the context switching overhead is irrelevant as most of the threads will be sleeping waiting for IO to occur anyway. And if you do use this module to create a worker pool that is doing busy work, limited to NUMCPU workers, you always have a free process that can handle IO events.
It is also important to understand Golang will create additional processes to handle Cgo invocations, like calling into the Mysql C library (if that is how it is implemented).
Context switching overhead only becomes a problem when you have more busy threads than you have cores - but the whole point of this module is you can limit the number of Goroutines executing a busy function.
Hope that helps.
@fcntl So I was thinking, going back to Jeffail's situation with the http requests coming in and spawning a goroutine per request. If we just simply put the busy work inside that goroutine instead of using a pool (assuming I'm not doing any syscalls or anything in the busy work that will create an OS thread per call because doing so could potentially crash the system I suppose if too many requests come in), it won't crash right? So I suppose in this context, it's not a matter of "I should use a pool because it won't crash the system", but more of a matter of at some point it's simply faster to have a set number of workers working on jobs as quickly as possible and have waiting jobs wait for their turn, than having a bunch of jobs working at the same time (which would be slower beyond a certain point than a pool due to context switching between goroutines etc.) Is my logic correct?
If we just simply put the busy work inside that goroutine instead of using a pool... it won't crash right?
Well it depends. If you spawn 1,000,000 goroutines all making backend connections to MySQL, then it could easily cause MySQL to crash (although there is a limit on the number of processes Golang will create for Cgo invocations).
If you spawn a lot of goroutines doing other busy work they might return very slowly, but should not crash the system, as GOMAXPROCS by default limits the number of OS processes to the number of CPU cores.
Think of this module as being useful to limit the number of goroutines executing a certain function, either straight busy work or perhaps making backend connections etc.
My other comments was that if you do expect to use a worker pool for busy work, to maximize utilization you are best off having NUMWORKERS=NUMCPU and GOMAXPROCS=NUMCPU+1, that way the workers each get a CPU core to keep busy, while there is a spare OS process ready to wake up and handle goroutines scheduled for IO work (like accepting more connections).
The example I gave on that other issue demonstrates the issue perfectly - try running it and seeing for yourself. With GOMAXPROCS=NUMCPU the http goroutine does not accept any more requests as the busy goroutines never yield control. Change it to GOMAXPROCS=NUMCPU+1 and the issue goes away and the http requests are handled normally. This does not cause big context switching overhead, as the IO process only wakes up rarely (in computer time). However, if you set NUMWORKERS>=GOMAXPROCS>NUMCPU then the context switching overhead would be terrible, as all these busy processes would be fighting over CPU cores.
If you are expecting your IO process to be very busy, e.g. handling 1000s requests/sec, then you could consider setting NUMWORKERS=NUMCPU-1 to give the IO process a core all to itself. Whether you set GOMAXPROCS to NUMCPUs or NUMCPU+1 doesn't really matter here, as IO processes dont use enough CPU usually to cause much context switching difficulties.
So thats how you should view this module - a way of separating the OS processes your busy goroutines run in vs your IO bound goroutines. You can even pin the worker goroutines to the particular core they are on, but this is not necessary, let the OS sort out that scheduling.
I see. Initially I was thinking maybe setting number of workers to more than number of CPUs (with GOMAXPROCS=NUMCPU) would be fine given that the mysql Go driver I was using uses a connection pool. Or would it just be better to be on the safe side to always have an available thread ready to accept incoming requests doing what you said (NUMWORKERS=NUMCPU and GOMAXPROCS=NUMCPU+1)?
@lazercorn if all you worker is doing is calling mysql, it is not really doing busy work, just sending calling through the mysql Go driver, so yes you could set the NUMWORKERS>NUMCPU with GOMAXPROCS=NUMCPU without much impact, as in realty all of your workers are mainly IO bound (yes even the mysql ones). However if your workers started doing busy work you would be in trouble due to context switching and scheduling starvation.
Or would it just be better to be on the safe side to always have an available thread ready to accept incoming requests doing what you said (NUMWORKERS=NUMCPU and GOMAXPROCS=NUMCPU+1)?
Well I guess it depends on WHY are you using this module. Are you using it to limit the amount of connections to MySQL? If so, you might also want to consider using a sempahore pattern like this: http://www.golangpatterns.info/concurrency/semaphores. But in this case may want more MySQL connections than the number of cores, in which case NUMWORKERS=NUMCPU will limit those connections, and there should not really be a relationship between how many concurrent queries MySQL can handle and the number of cores in the calling CPU.
On the other hand, if you are using it to limit the number of busy goroutines, and you want to maximize utilization while keep IO threads responsive, then yes NUMWORKERS=NUMCPU and GOMAXPROCS=NUMCPU+1 is the best option.
You need to clarify your reason for using this module in the first place and what you hope it will achieve for you.
My use case for this library would be more of limiting the number of requests sent to the mysql backend than doing any busy work because as you've said most of the stuff I do is IO bound and there really isn't anything "busy" I would say with regards to what I'm doing after I get back the results from the database because all I'm doing is some if else checks on the results. I most likely will just be experimenting with the number of workers because it seems for my case it would be better to have more workers than number of CPUs due to how I'm mainly just doing IO and nothing busy with the results.
In another scenario, what if instead of the mysql database being remote, it was local. Then instead of it being IO bound would it be more CPU bound then?
@lazercorn no all modern versions of mySQL use a client/server architecture so it's always going to be IO bound workers, even on localhost.
In your case then I would probably set NUMWORKERS>NUMCPU and GOMAXPROCS=NUMCPU. You don't need to worry about a dedicated IO thread as all of your goroutines are IO bound. GOMAXPROCS=NUMCPU+1 only makes sense when your workers are CPU bound and you probably don't want NUMWORKERS to be greater than the number of cores.
When I mentioned CPU bound I was referring to how both the local mysql database and the Go webserver would be competing for CPU. Is it possible at some point it would be CPU bound?
Well you obviously need to consider load on your server, but if all Go is doing is IO jobs they won't really compete with Mysql. The significance of CPU bound jobs is there is no point having more of them than CPU cores, but that is not the case as with IO bound jobs.
Gotcha. Thanks for taking the time to answer all my questions. Really learned a lot! 👍