refreshing secrets is slow
Closed this issue · 4 comments
Right now knife vault refresh foo bar takes a long time if the search for foo/bar hits a lot of nodes (hundreds to about a thousand in my test).
From my analysis, this primarily comes from two sources:
- Downloading all the node objects take a long time. About two minutes on my test connection (hosted Chef, I'm in Europe on a DSL line). As we don't actually need the full node objects, we can use a partial search and just ask for the host name.
- Downloading the client keys is dominated massively by latency. I see two ways this can be solved: use threads or construct a big search query to download all the public keys in one go; basically doing
.search(:client, "name:foo OR name:bar OR name:baz …).
Resurrecting this very old ticket.
We have exactly the same issue and have implemented a few patches to solve that issue:
Once all these PR will be merged, cost of refresh should be:
- cost of search to the chef server
- cost of encrypting symetrical key for new nodes only
- saving 1 data bag item (xxx_keys) to the chef server
You can read the detail of improvements on each PR.
Edit, with all these improvements (starting from chef-vault < 3.0) our total refresh time (to refresh all secrets when search_query result has not changed + update secrets to include all admins) has gone from >3h to 20minutes.
There is still easy points on the way updates are done when changing only the admin list so I think it would be doable to win a few more minutes there.
I think we can now close this issue and reopen a new one if some obvious speed issue is found.