/batch-demo

A demo illustrating the use of Spring batch for use with AWS batch

Primary LanguageJava

AWS Batch and spring Batch demo

This small project is a simple Spring batch application that reads a CSV file from AWS S3 and parses it using Open CSV multithreaded parser. It may run on AWS Batch once package as a docker container (see Dockerfile)

With 40 cores and enough RAM it parses 29Millions rows (10Gb of data) in less than 5 minutes.