Pinned Repositories
artwork
Open-licensed artwork related to Data Together
datatogether
:checkered_flag: Start here! Discussion for Data Together: Building a better future for data
identity
User & Identity Management server
reading_datatogether
📚 Monthly reading group for Data Together
research
📚 A compilation of research relevant to Data Together's efforts tackling the general problem of data resilience & interactivity
sentry
Parallelized web crawler written in Golang
warc
Golang WARC (Web ARChive) Library
webapp
Web application to allow users to add content metadata about crawled resources
website
A static-generated website to introduce the Data Together project, built with Hugo.
xmp
Golang package for parsing Extensible Metadata Platform (XMP) documents
Data Together's Repositories
datatogether/research
📚 A compilation of research relevant to Data Together's efforts tackling the general problem of data resilience & interactivity
datatogether/datatogether
:checkered_flag: Start here! Discussion for Data Together: Building a better future for data
datatogether/reading_datatogether
📚 Monthly reading group for Data Together
datatogether/warc
Golang WARC (Web ARChive) Library
datatogether/sentry
Parallelized web crawler written in Golang
datatogether/webapp
Web application to allow users to add content metadata about crawled resources
datatogether/website
A static-generated website to introduce the Data Together project, built with Hugo.
datatogether/xmp
Golang package for parsing Extensible Metadata Platform (XMP) documents
datatogether/dt
command line tool for storing data together on the distributed web
datatogether/roadmap
Coordinating technical work & roadmapping additional services
datatogether/archivertools
Python package for scraping websites into the Data Together pipeline via morph.io
datatogether/cdxj
Golang package implementing the CDXJ file format used by OpenWayback 3.0.0+ to index web archive contents
datatogether/core
Core Archive Model Definitions
datatogether/ffi
Golang package for making sensible guesses about file formats from Url strings
datatogether/extract_href
Command line tool for extracting urls from a HTML web page using a jquery-style selector
datatogether/linked_data
Golang package for working with linked data structures, an implementation of the W3C Data Catalog Vocabulary (DCAT)
datatogether/patchbay
Websockets-based API backend for our react and redux webapp.
datatogether/pdf
Library that extracts xmp metadata from a PDF document.
datatogether/sql_datastore
Experimental Golang implementation of the ipfs datastore interface for sql databases.
datatogether/artwork
Open-licensed artwork related to Data Together
datatogether/sql_util
Golang package that provides utils for working with dotsql and postgres
datatogether/archive
golang package for creating/working with warc & cdxj archives
datatogether/content
Service for serving archived content stored on amazon S3.
datatogether/coverage
Project for visualizing the status of digital data archiving efforts across various data repositories
datatogether/resources
extract urls of dependant files for displaying a web page
datatogether/task_mgmt
Service for managing & executing archiving tasks written in Golang
datatogether/api
Serves the Data Together JSON API
datatogether/learning
Learning materials for Data Together
datatogether/OS_deployment
datatogether/rewrite
modify the contents of web-related content types for archival purposes