Library which can traverse archived file (using libarchive under the hood) and yields IO like object on each file entry inside it for further streamline processing. Stress-tested via processing 500 MB big archive (~10GB big xml file when uncompressed) and it worked quite good. No memory peaks during the process were observed.
Note: libarchive have to be pre-installed and available on the host system
Add this line to your application's Gemfile:
gem 'archive_io'
And then execute:
$ bundle
Or install it yourself as:
$ gem install archive_io
Simple usage:
archive = ArchiveIO.open("archive.7z")
archive.each do |cursor|
puts cursor.pathname # prints out pathname inside archive
puts cursor.read(10) # prints out beginning of each file
end
archive.close
This library can come in handy if you want to process huge xml files reading straight from the archive without uncompressing it and works nicely together with Nokogiri::XML::Reader
and can be used as follows:
archive = ArchiveIO.open("archive.7z")
archive.select("*.xml") do |cursor|
Nokogiri::XML::Reader(cursor).each do |xml_node|
# your custom xml processing logic goes here
end
end
archive.close
After checking out the repo, run bundle install
to install dependencies. You can also run bin/console
for an interactive prompt that will allow you to experiment.
To release a new version, update the version number in version.rb
, and then run bundle exec rake release
, which will create a git tag for the version, push git commits and tags, and push the .gem
file to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/AMekss/archive_io.