/spring-batch-s3-sample-app

Sample Spring batch job for processing S3 objects one by one.

Primary LanguageJavaMIT LicenseMIT

spring-batch-s3-sample-app

Reads CSV files stored S3 folder in a bucket one by one and process them (Like calling a third party API) and writes the responses to another subdirectory called responses and then process the responses to generate a summary file.

S3 Configuration

Please update application.properties with your AWS access key details

S3 folder structure

  • Requests: s3://bucket-name/folder-path/requests
  • Processed requests: s3://bucket-name/folder-path/processed (once request file is processed it will be moved to this folder)
  • Responses: s3://bucket-name/folder-path/responses (this will be created)
  • Summary file: s3://bucket-name/folder-path/summary.txt

Rest API

This is for triggering and getting status of the jobs once the application is started.