Get all 397,000 series from FRED, is that possible?
yzhao1216 opened this issue · 2 comments
Hi
I read the blog you write, it is really helpful.
I am thinking about get all the data observations by iterating through all the series_ids, like this:
-------https://api.stlouisfed.org/fred/series/observations?series_id=[for i in all series_ids]&api_key=************---------
Do you think its possible to do so? If it is , is there any good methods you recommend?
Thanks
Yiwei
Why on earth would you want to create a local copy of the whole FRED database? It's not going anywhere and it's reasonably quick, just call it as you need it.
I mean, you could do something like:
fred = fredapi.Fred(api_key=***)
for i in series_ids:
fred.get_series(i).to_pickle("%s.pkl" % i)
But the likelihood that the API would kick you out before it finished and block your API key is really, really high, for good reason-- you're basically doing a DoS attack on them.
It used to be possible to download all the series as a single zip file from this no-longer-working URL. They changed that about a year ago. When asked, they said you can still get zip files for each of the (~8) major categories here. You'll need to click through to find the exact URLs, and it will involve choosing between csv and txt formats.