truefoundry/secretsfoundry

Make the process of fetching secrets from loaders parallel instead of serial

Closed this issue · 0 comments

We currently fetch all the secrets from loaders in SecretsFoundry.ts in a sequential manner. You can check this in SecretsFoundry.ts in resolveVariables function - where we do await for resolving every sngle variable. This causes the parsing of variables to be very slow since if there are 10+ env variables to be loaded, it means 10 network requests serially.

However fetching all of the secrets in parallel is not trivial since there might be dependencies between the variables. So this involves creating a dependency graph of promises that need to be resolved - almost like a topological graph.