AbsaOSS/atum

Integration tests for S3-based routines

Opened this issue · 1 comments

While #37 solves funcioning of Atum on the S3 persistence layer, it would be quite beneficial in terms of project quality to cover these routines via integration tests as well. (There are unit tests present, but still)

Couple of options come to mind to be able to write sufficient integration tests:

  • localstack, e.g. in form of a testcontainer module or otherwise
  • using a specific library just for s3, maybe Findify's s3mock (This is currently written for SDK v1, but we are using SDK v2 already) or a similar alternative
  • running against a real dedicated S3 bucket set up specifically for S3 integrations tests
  • ?

@wajda how was this approached in Spline in the end? Did you guys use testcontainer with localstack?