Finalize and test customcrawls data together endpoint
Opened this issue · 2 comments
ebenp commented
Associated with datatogether/roadmap#63
jeffreyliu commented
TODO: Create a test scraper on morph that has saved various test files of different types onto the morph db tables, and then ensure that they are replicated correctly on the DT side.
test files - feel free to add suggestions: an image, text file, large binary, duplicate files, compressed file
b5 commented
yo yo! So quick update on this, we now have an operational endpoint at api.archivers.co/customcrawls read for testing!