Pinned Repositories
awesome-web-archiving
An Awesome List for getting started with web archiving
heritrix3
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
jwarc
Java library for reading and writing WARC files with a typed API
openwayback
The OpenWayback Development
robustlinks
Links on the web break all the time, robustify them!
twittervane
Using social media to steer web archiving and curation.
urlcanon
url canonicalization library for python and java
warc-specifications
Centralised repository for WARC usage specifications.
warc2html
Converts WARC files to static HTML
webarchive-commons
Common web archive utility code.
IIPC's Repositories
iipc/awesome-web-archiving
An Awesome List for getting started with web archiving
iipc/openwayback
The OpenWayback Development
iipc/warc-specifications
Centralised repository for WARC usage specifications.
iipc/robustlinks
Links on the web break all the time, robustify them!
iipc/webarchive-commons
Common web archive utility code.
iipc/jwarc
Java library for reading and writing WARC files with a typed API
iipc/warc2html
Converts WARC files to static HTML
iipc/urlcanon
url canonicalization library for python and java
iipc/twittervane
Using social media to steer web archiving and curation.
iipc/heritrix3
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
iipc/iipc.github.io
IIPC Open Development
iipc/openwayback-access-control
web access control (exclusion oracle) tools for optional use with wayback machine
iipc/warcaroo
iipc/training
Inventory of Web Archiving Training Resources
iipc/archive-crawler-yg-archive
An 'archive' of the Yahoo-hosted archive-crawler group
iipc/logtrix
Java library/tool for parsing and summarising Heritrix crawl logs
iipc/openwayback-sample-overlay
Sample Wayback Config using OpenWayback
iipc/qa2019
Resources for the 2019 IIPC QA hackathon
iipc/cdx-cli
Command line utility for working with CDX files
iipc/travis
Shared config for Travis CI for IIPC.
iipc/crawl-recipes
A place to share practical bits of crawling experiences
iipc/heritrix-dependencies
Dependencies needed to build Heritrix that aren't in Maven Central
iipc/iipc-parent
IIPC Parent POM
iipc/javaswf
Fork of JavaSWF2 for building Heritrix