internetarchive/dweb-mirror

crawl fails to get meta.xml

Closed this issue · 1 comments

crawl all opens commute_meta.xml.part but fails silently somewhere before it saves the file.

See #140 which will be wrong till this fixed

  • Repeatable with curl -Lv http://localhost:4244/download/commute/commute_meta.xml

Fixed, was a problem with dweb-cors which passed on a content-encoding header: zip, DM couldnt decode this since it wasnt expecting zip.