/fszipfilesstore

Zip-backed file storage for scrapy FilesPipeline module

Primary LanguagePythonApache License 2.0Apache-2.0

FSZipFilesStore

Zip-backed file storage for scrapy FilesPipeline module. Files are downloaded and compressed in-memory and written directly to a zip-file. A compression ratio of ~16 is achieved with standard HTML/XML files.