[BUG] apache.vhosts.standard (and potentially others) generate MemoryError
ixs opened this issue · 9 comments
I have a setup here that has about 200 vhosts using the apache.vhosts.standard state.
state.apply calls fail with a MemoryError and slsutil.renderer generates about 1GB of json...
This seems due to the full apache variable being passed multiple times as context thus exponentially increasing memory consumption.
Is an example where the full apache variable is passed in twice...
Thanks for the report, @ixs. I add that the entries under the context block need to be indented by another 2 spaces as well:
apache-formula/apache/config/vhosts/standard.sls
Lines 22 to 26 in e2e1be1
Meaning:
- context:
apache: {{ apache|json }}
id: {{ id|json }}
...@noelmcloughlin Could you have a look at this, please?
No need. I have something locally which I'm testing already as PR.
How did I become a code owner ;-) We need to encourage issue raisers to contribute PRs too. Okay, the value
apache: {{ apache|json }}was added a few years ago, I can raise a PR.
@noelmcloughlin Well, I usually check the blame to see who last changed the section which has caused the regression and then ping accordingly. It's nothing personal! However, you're probably going to get more pings when there's wholescale changes made to formulas!
#298 is the PR... Works fine locally. We're still passing way too much information per templating call but at least we're not using exponentially more memory...
PR merged. closing. Quick turnaround, thanks guys.
Thanks for the issue. Lets keep it open for a while.
I raised #299 as follow-up. There may be more but this PR was obvious.
👍🏻
There are more cases where full dicts are passed,but the bad ones are where it's done per iteration like in vhosts.