Process data from object storage
Opened this issue · 0 comments
vinjana commented
Preferentially S3.
- Stream directly into pipes e.g. s3cmd. Combine with decompression
- How to determine the compression of input data? Add extra compression determination step?
- Write all output data to target bucket
Problems:
- Roddy cannot yet accept remote files (Are they present? Roddy checks them on FS in the moment but there is no support for remote file checking in S3.
We may first have to implement some basic support in Roddy. See TheRoddyWMS/Roddy#331