S3Query
S3Query opens file or files in s3 file system as text file, will perform unzip if necessary. It also handles hadoop liked part-* files and treat all files within a directory as one single file.
Install
pip install s3query
Example
>>> from s3query import s3open
>>> import sys
>>>
>>> with s3open("s3://s3query/data/testgz/") as fgz:
... for line in fgz:
... sys.stdout.write(line)
...
one
two
three
four
five
six
seven
eight
nine
>>> with s3open("s3://s3query/data/test/") as fplain:
... for line in fplain:
... sys.stdout.write(line)
...
one
two
three
four
five
six
seven
eight
nine