Archive Team

We Are Going To Rescue Your Shit

The Internet

Pinned Repositories

ArchiveBot
ArchiveBot, an IRC bot for archiving websites
Language:Python400 25 28477
grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Language:Python1.5k 42 210150
IA.BAK
We back up a lot of stuff from around the web; now it's time to back up the Internet Archive, just in case.
Language:Shell89 25 3522
NewsGrabber
Grabbing all news.
Language:Python62 17 5032
parler-grab
Archiving Parler.
Language:Lua227 30 127
seesaw-kit
Making a reusable toolkit for writing seesaw scripts
Language:Python72 9 12334
terroroftinytown
URLTeam's second generation of URL shortener archiving tools
Language:Python80 18 4715
warrior-dockerfile
A Dockerfile for the ArchiveTeam Warrior
Language:Shell407 20 5264
wget-lua
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
Language:C129 20 1816
wpull
Wget-compatible web downloader and crawler.
Language:HTML582 22 45878

Archive Team's Repositories

ArchiveTeam/ArchiveBot
ArchiveBot, an IRC bot for archiving websites
Language:Python400 25 28477
ArchiveTeam/usgovernment-grab
Archiving parts of the US government.
Language:Lua27 9 33
ArchiveTeam/warrior4-vm
Warrior virtual machine appliance (version 4)
Language:Rust27 8 121
ArchiveTeam/urls-grab
Archiving URLs (outlinks) from a variety of sources.
Language:Lua21 10 106
ArchiveTeam/megawarc
Nondestructive warc-in-tar to warc conversion
Language:Python8 13 64
ArchiveTeam/archiveteam-megawarc-factory
Some scripts to process ArchiveTeam uploads
Language:Shell6 11 86
ArchiveTeam/goo-gl-grab
Archiving goo.gl, the Google URL Shortener.
Language:Lua2 5 0
ArchiveTeam/warrior-hq
Warrior HQ API
Language:HTML2 2 3
ArchiveTeam/goo-gl-items
Managing items for goo-gl-grab.
Language:Roff1 5 0
ArchiveTeam/voiceofamerica-grab
Archiving Voice of America (all sites).
Language:Lua1 6 21
ArchiveTeam/voiceofamerica-items
Managing items for voiceofamerica-grab.
1 6 0
ArchiveTeam/zhubai-grab
Archiving 竹白 (Zhubai).
Language:Lua1 6 0
ArchiveTeam/arzon-grab
Archiving arzon.jp.
6 0
ArchiveTeam/arzon-items
Managing items for arzon-grab.
ArchiveTeam/dailymotion-grab
Archiving Dailymotion.
6 0
ArchiveTeam/dailymotion-items
Managing items for dailymotion-grab.
6 0
ArchiveTeam/grab-tools
Tools for *-grab projects.
6 0
ArchiveTeam/ipcc-items
Managing items for ipcc-grab.
6 0
ArchiveTeam/ipccdata-grab
Archiving ipcc-data.org.
6 0
ArchiveTeam/livestream-grab
Archiving livestream.com.
Language:Lua6 12
ArchiveTeam/mahonoiland-grab
Archiving 魔法のiらんど (Maho-no iLand).
6 0
ArchiveTeam/plimit
Redis based parallelism limiter
Language:Go7 0
ArchiveTeam/radiofreeeurope-grab
Archiving Radio Free Europe/Radio Liberty (aka RFERL aka RFE/RL)
ArchiveTeam/radiofreeeurope-items
Managing items for radiofreeeurope-grab.
ArchiveTeam/retrospring-grab
Archiving Retrospring.
Language:Lua6 0
ArchiveTeam/sonetblog-grab
Archiving SSブログ" (So-net Blog).
6 0
ArchiveTeam/torrentgalaxy-grab
Archiving TorrentGalaxy.
6 0
ArchiveTeam/torrentgalaxy-items
Managing items for torrentgalaxy-grab.
6 0
ArchiveTeam/usgovernment-items
Managing items for usgovernment-grab.
ArchiveTeam/zhubai-items
Managing items for zhubai-grab.
Language:Python6 2