Do you have suggestions for reading many csv files
Closed this issue · 2 comments
alexhallam commented
I would like to do something like a glob so I can read many csv files from a directory. I was curious was kind of solutions paws has for this. Currently, when I run get_object
I can't pass something like dir/*.csv
as a key. I have to pass dir/file1.csv
.
davidkretch commented
There is no multiple download as far as I know. I think the most straightforward way would be to call list_objects_v2
, filter the result however you want, and download each one at a time. It might looks something like this (not tested). Hopefully this helps.
objects <- s3$list_objects_v2(Bucket = "my-bucket")
to_download <- Filter(function(x) grepl(".*\\.csv", x), a$Contents)
for (file in to_download) {
...
}
Hopefully that helps
davidkretch commented
Closing. Let us know if this is still an issue!