goto/guardian

optimize table fetch from bigquery

Closed this issue · 0 comments

Summary
Currently, in bigquery GetResources call, we do a mutex lock before we do append to the resources slice. But since we do a defer unlock, the entire code block is blocking other goroutines. So even though we have errgroup with limit of 20, the code is running sequentially only due to the defer unlock.

This can be made faster by just locking and unlocking the specific non-concurrent code.