Archive Team

We Are Going To Rescue Your Shit

The Internet

Pinned Repositories

ArchiveBot
ArchiveBot, an IRC bot for archiving websites
Language:Python401 22 28479
grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Language:Python1.5k 39 210150
IA.BAK
We back up a lot of stuff from around the web; now it's time to back up the Internet Archive, just in case.
Language:Shell92 25 3522
parler-grab
Archiving Parler.
Language:Lua227 30 125
seesaw-kit
Making a reusable toolkit for writing seesaw scripts
Language:Python72 9 12334
Ubuntu-Warrior
Scripts to build and boot warrior virtual machine containing Docker
Language:Shell121 15 169
warrior-dockerfile
A Dockerfile for the ArchiveTeam Warrior
Language:Shell414 18 5365
warrior4-vm
Warrior virtual machine appliance (version 4)
Language:Rust49 8 121
wget-lua
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
Language:C129 20 1917
wpull
Wget-compatible web downloader and crawler.
Language:HTML595 22 46184

Archive Team's Repositories

ArchiveTeam/warrior-dockerfile
A Dockerfile for the ArchiveTeam Warrior
Language:Shell414 18 5365
ArchiveTeam/ArchiveBot
ArchiveBot, an IRC bot for archiving websites
Language:Python401 22 28479
ArchiveTeam/wget-lua
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
Language:C129 20 1917
ArchiveTeam/terroroftinytown
URLTeam's second generation of URL shortener archiving tools
Language:Python79 18 4715
ArchiveTeam/imgur-grab
Archiving imgur.
Language:Lua61 18 44
ArchiveTeam/ludios_wpull
wpull fork with fixes and faster parsing using html5-parser; used by grab-site; should go away when wpull is similarly improved
Language:HTML30 6 208
ArchiveTeam/usgovernment-grab
Archiving parts of the US government.
Language:Lua29 9 34
ArchiveTeam/youtube-grab
Archiving all metadata from YouTube (everything except videos themselves due to size)
Language:Lua28 9 132
ArchiveTeam/urls-grab
Archiving URLs (outlinks) from a variety of sources.
Language:Lua23 10 118
ArchiveTeam/telegram-grab
Archiving public telegram messages.
Language:Lua14 10 129
ArchiveTeam/pastebin-grab
Archiving pastebin
Language:Python11 6 14
ArchiveTeam/github-grab
Archiving GitHub
Language:Lua10 7 87
ArchiveTeam/megawarc
Nondestructive warc-in-tar to warc conversion
Language:Python8 13 64
ArchiveTeam/goo-gl-grab
Archiving goo.gl, the Google URL Shortener.
Language:Lua7 6 32
ArchiveTeam/mediafire-grab
Archiving mediafire.com URLs.
Language:Lua7 6 17
ArchiveTeam/grab-base-df
Base Dockerfile for warrior project grab scripts
Language:Dockerfile5 15 54
ArchiveTeam/glitch-grab
Archiving Glitch.
Language:Lua2 0 0
ArchiveTeam/.github
ArchiveTeam/itchio-minimal-grab
Archiving at risk data from itch.io.
Language:Lua
ArchiveTeam/itchio-minimal-items
Managing items for itchio-minimal-grab.
ArchiveTeam/livestream-grab
Archiving livestream.com.
Language:Lua6 13
ArchiveTeam/oshietegoo-grab
Archiving 教えて!goo.
Language:Lua2
ArchiveTeam/oshietegoo-items
Managing items for oshietegoo-grab.
ArchiveTeam/peingquestionbox-grab
Archiving peing.net.
Language:Lua
ArchiveTeam/peingquestionbox-items
Managing items for peingquestionbox-grab.
ArchiveTeam/tistory-grab
Archiving Tistory.
Language:Lua
ArchiveTeam/tistory-items
Managing items for tistory-grab.
ArchiveTeam/twitch-items
Managing items for twitch-grab.
Language:Python6 0
ArchiveTeam/typepad-grab
Archiving Typepad.
Language:Lua
ArchiveTeam/typepad-items
Managing items for typepad-grab.
0 0