Mastercard: dates in file names and historical data check
Closed this issue · 2 comments
objornsson commented
- Change recipe to not create new date folders, but to put all data in all_data
- Change recipe to create .csvs instead of .zips
- Change recipe to overwrite in latest folder, referencing version history of Mastercard recipe or other recipes like StreetEasy. Only one file should be in latest folder at one time
- Move new csvs from latest to all_data
After this, Andrew to delete files with old naming conventions and delete old date folders.
SteveScott commented
The recipe should no longer create new date folders. The format is now unzipped. There is no delete feature in the existing sharepoint code: you can only overwrite with the same filename. This is how StreetEasy does it. The file is now mastercard_latest.csv. This will be placed in the 'Latest' folder.
SteveScott commented
CSVs moved from Latest to All_Data