TeaInSpace/ame

e2e tests

jmintb opened this issue · 0 comments

AME will require a large suite of e2e tests to ensure that everything actually works. Up until now we have included such tests inside the cli module and hacked it together relying on external services such as github to get tests implemented fast.

This is not scalable or desirable going forward. Therefore some time must be invested in creating e2e test infrastructure within the project.

Acceptance Criteria

  • Everything is self contained except when explicitly testing against external services such as Github or Azure blob storage. This means that anything which can be deployed along side the tests must do so. This ensure that tests can have complete control over the environment and avoids excessive dependence on external services.
  • For tests against external services there must be a way to create and destroy the required environment within that external service.
  • Developers should have clear documentation on how and when to write such tests.
  • 2e2 tests can be run against a matrix of different environments and dependencies.