/hxl-spider

Simple spider to crawl HXL datasets on the Humanitarian Data Exchange and report stats.

Primary LanguagePython

Script to crawl HXL datasets on HDX and collect statistics

Prerequisites

  • Python3
  • the ckanapi module
  • the libhxl module
  • an account on a CKAN instance

Instructions

  1. Copy the file config.py.TEMPLATE to config.py and fill in the fields
  2. Execute the command python3 crawl-hxl.py