Archive Team

We Are Going To Rescue Your Shit

The Internet

Pinned Repositories

ArchiveBot
ArchiveBot, an IRC bot for archiving websites
Language:Python347 28 28069
grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Language:Python1.3k 40 205125
IA.BAK
We back up a lot of stuff from around the web; now it's time to back up the Internet Archive, just in case.
Language:Shell86 26 3522
NewsGrabber
Grabbing all news.
Language:Python61 18 5032
parler-grab
Archiving Parler.
Language:Lua227 31 127
seesaw-kit
Making a reusable toolkit for writing seesaw scripts
Language:Python68 10 11530
terroroftinytown
URLTeam's second generation of URL shortener archiving tools
Language:Python69 20 4615
Ubuntu-Warrior
Scripts to build and boot warrior virtual machine containing Docker
Language:Shell109 16 1610
warrior-dockerfile
A Dockerfile for the ArchiveTeam Warrior
Language:Shell271 19 4557
wpull
Wget-compatible web downloader and crawler.
Language:HTML540 23 45277

Archive Team's Repositories

ArchiveTeam/grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Language:Python1.3k 40 205125
ArchiveTeam/wpull
Wget-compatible web downloader and crawler.
Language:HTML540 23 45277
ArchiveTeam/ArchiveBot
ArchiveBot, an IRC bot for archiving websites
Language:Python347 28 28069
ArchiveTeam/warrior-dockerfile
A Dockerfile for the ArchiveTeam Warrior
Language:Shell271 19 4557
ArchiveTeam/wget-lua
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
Language:C84 20 1614
ArchiveTeam/imgur-grab
Archiving imgur.
Language:Lua65 21 43
ArchiveTeam/reddit-grab
Grabbing everything from reddit.
Language:Lua60 18 1112
ArchiveTeam/terroroftinytown-client-grab
The Seesaw pipeline grab script for the URLTeam (terroroftinytown) project
Language:Python26 15 511
ArchiveTeam/ludios_wpull
wpull fork with fixes and faster parsing using html5-parser; used by grab-site; should go away when wpull is similarly improved
Language:HTML25 6 205
ArchiveTeam/youtube-grab
Archiving all metadata from YouTube (everything except videos themselves due to size)
Language:Lua20 11 102
ArchiveTeam/urls-grab
Archiving URLs (outlinks) from a variety of sources.
Language:Lua14 11 95
ArchiveTeam/telegram-grab
Archiving public telegram messages.
Language:Lua13 11 108
ArchiveTeam/warrior4-vm
Warrior virtual machine appliance (version 4)
Language:Rust12 9 61
ArchiveTeam/pastebin-grab
Archiving pastebin
Language:Python9 7 14
ArchiveTeam/subscene-grab
Archiving Subscene.
Language:Lua74
ArchiveTeam/mediafire-grab
Archiving mediafire.com URLs.
Language:Lua6 7 15
ArchiveTeam/grab-base-df
Base Dockerfile for warrior project grab scripts
Language:Dockerfile4 16 33
ArchiveTeam/urls-sources
Sources for urls-grab.
Language:Python4 7 178
ArchiveTeam/roblox-marketplace-comments-grab
Archiving comments from the Roblox Marketplace
Language:Lua1
ArchiveTeam/subscene-items
Managing items for subscene-grab.
1
ArchiveTeam/urls-tor-grab
Archiving some .onion URLs.
Language:Python1
ArchiveTeam/deviantart-grab
Archiving part of DeviantArt.
Language:Lua
ArchiveTeam/deviantart-items
Managing items for deviantart-grab.
ArchiveTeam/postnews-grab
Archiving post.news.
Language:Lua
ArchiveTeam/postnews-items
Managing items for postnews-grab.
Language:Roff
ArchiveTeam/roblox-marketplace-comments-items
Managing items for roblox-marketplace-comments-grab.
ArchiveTeam/taringa-grab
Archiving taringa.net.
Language:Lua6 0
ArchiveTeam/taringa-items
Managing items for taringa-grab.
Language:Python6 0
ArchiveTeam/vbox7-grab
Archiving vbox7.
Language:Lua5 0
ArchiveTeam/vbox7-items
Managing items for vbox7-grab.
6 0