How can Scandownloader handle LARGE scan results?

Question

How can Scandownloader handle LARGE scan results?

Closed this issue 8 years ago · 2 comments

How can the script be modified to pipe the scan result directly to a zip file on disk without staging in memory. We have scan result files exceeding 1 GB zipped and the script blows memory even with 16GB RAM.

Answer 1 · 2016-04-12T21:43:10.000Z

Are you referring to?:

https://github.com/SteveMcGrath/pySecurityCenter/blob/master/examples/sc5/download_scans/downloader.py

Answer 2 · 2016-04-12T21:59:07.000Z

I'd recommend replacing the existing post and write code with something closer to this:

http://stackoverflow.com/questions/14114729/save-a-large-file-using-the-python-requests-library?answertab=votes#tab-top