Get all 397,000 series from FRED, is that possible?

[yzhao1216]

Hi
I read the blog you write, it is really helpful.
I am thinking about get all the data observations by iterating through all the series_ids, like this:
-------https://api.stlouisfed.org/fred/series/observations?series_id=[for i in all series_ids]&api_key=************---------
Do you think its possible to do so? If it is , is there any good methods you recommend?

Thanks
Yiwei

[Liam3851]

Why on earth would you want to create a local copy of the whole FRED database? It's not going anywhere and it's reasonably quick, just call it as you need it.

I mean, you could do something like:

fred = fredapi.Fred(api_key=***)

for i in series_ids:
    fred.get_series(i).to_pickle("%s.pkl" % i)

But the likelihood that the API would kick you out before it finished and block your API key is really, really high, for good reason-- you're basically doing a DoS attack on them.

[jbrockmendel]

It used to be possible to download all the series as a single zip file from this no-longer-working URL. They changed that about a year ago. When asked, they said you can still get zip files for each of the (~8) major categories here. You'll need to click through to find the exact URLs, and it will involve choosing between csv and txt formats.