alan-turing-institute/AIrsenal

Data Query (Not an Issue)

Closed this issue · 4 comments

RajeshTailor1 commented

How is the data imported into the database? I can see a few JSON files and CSV files is most of the key data imported from the CSV files (where they exist) and also the API requests?

nbarlowATI commented

Hi @RajeshTailor1 , yes, you're right, most of the key data (almost all of the data for the current season) is from the FPL API, and the requests are made by the FPLDataFetcher defined in airsenal/framework/data_fetcher.py.
The csv files etc. are mostly for historic data, with the exception of the "teams" one, which lists the 20 teams in the league (there isn't an API endpoint for that...)
Hope that helps!

RajeshTailor1 commented

Thanks for confirming. Does the FPL API data request go straight into the database? Or is it written to a CSV file first?

nbarlowATI commented

It goes straight into the database, with no intermediate SVG. The code for this is mostly in airsenal/scripts/fill_XYZ_table.ppy, where there are often functions to fill historical data from csv, and current-season data from the API, but all being written into the same database table

  • 👍1
RajeshTailor1 commented

Thanks.