awslabs/ml-io

Example Needed: Reading Numpy files from SageMakerPipe

system123 opened this issue · 0 comments

My dataset consists of a collection of .npy files stored in S3. As the dataset is relatively large I am wanting to access these from SageMaker using the SageMakerPipe source, however, it doesn't seem that there is a DatasetReader which supports plain Numpy files. Nor does it seem that the ImageDatasetReader could be used for this as it only supports JPG, PNG files.

Is there a way to achieve this? or does the library need to be extended?