ydup/crawling-data-from-fred

:tada: How to crawl the FRED big data in a rapid and methodical way

Crawling Data from FRED

This is a tutorial about how to crawl the FRED big data in a rapid and methodical way

• A interactive linear tree figure of the category structure of the FRED is in the webpage, with the help of this blog

• A round tree figure is in the webpage, with the help of the Observable notebook

• A jupyter notebook of the strategy implemention.

• Tree of the category crawler and json file generator

  • crawlingTree.py
  • cateTree.csv
  • jsonFile.json

• Download the top-10 popular data in the single thread way

  • singleThread.py

• Download the top-10 popular data in the multi thread way

  • multiThread.py