fossasia/susi_server

Add integration tests for basic skill features

spotlesscoder opened this issue · 2 comments

As discovered on CCCamp 2019, there might be issues with server side components sometimes.

When fixing those or adding new features to the SUSI.AI skill language, it should be made sure that existing functionality is not broken.

For that, there should be a CI system which executes common skills using basic functionality such as basic conversations, calling external APIs (which should be mocked during the test), and checks the results.

I think that can be done. We have a python api client that can be cloned during ci testing, and send of a set of test queries and check for correct answers.

I try to look into it ...

@mariobehling @Orbiter THere seems to be something broken in the .travis.yml file:

      env:
        global:
          - GH_REF: github.com/fossasia/susi_server.git
          - secure: ...

that doesn't work, it needs to be GH_REF=... etc.

Then for tests, there are obviously some test run

      after_success:
        - if [ -e ./gradlew ]; then ./gradlew test jacocoTestReport;else gradle test jacocoTestReport;fi

but obviously these tests don't really do anything useful, at least the travis-ci output doesn't indicate any activity.

I guess that is the place that needs fixing, because the test directory already contains quite some test cases.