Memory usage awareness: recommend late import
sparrowt opened this issue · 0 comments
While profiling memory usage I noticed that importing zxcvbn
was causing ~12mb of memory allocation:
This is not necessarily surprising but I thought I'd mention it here in case there was any obvious way to reduce it.
According to the profiling there was:
- 5 MB from the top level importing of
site-packages/zxcvbn/frequency_lists.py:4
- frequency_lists.py itself is 775 KB though importing it seems to result in a much larger memory increase (c.f. https://pythonspeed.com/articles/python-object-memory/)
- 7 MB from
build_ranked_dict.<locals>.<dictcomp>
while importingsite-packages/zxcvbn/matching.py:10
- matching.py imports frequency_lists and processes them into
RANKED_DICTIONARIES
which causes the extra usage
- matching.py imports frequency_lists and processes them into
I'm not necessarily suggesting that the behaviour of this package be changed, though I guess it would be possible to make the import very cheap and lazily do all the above effort only the first time it is required.
In the mean time though, I could easily get an equivalent benefit by moving my import of zxcvbn
to happen only inside the request handler which needs it, so that most of the python processes handling requests didn't need to bother, unless they received a request involving a user attempting to set a password.