hyperboria/peers

Better warnings in tests

thefinn93 opened this issue · 0 comments

Right now tests.py can pass with warnings, but those warnings are only visible in the Travis CI web interface or by manually running tests.py. I'd like to investigate better ways to showing these warnings. Options that come to mind:

  • Expanding tests.py to post the warnings on GitHub. eg. if the warning is thrown when running a test for a pull request, comment directly in pull request thread like landscape-bot does, and if it was committed directly to the repo open an issue or something.
  • Build our own test server that listens for requests from a GitHub webhook to be notified of new pulls or commits, then uses the GitHub Status API to communicate the warning (eg, state=success and description=Not all recommended fields present or something, possibly with a target_url that could better explain which fields are missing and what needs to be fixed)
  • Build a server that listens for webhooks for pull requests like above, but use GitHub's tagging system to tag pull requests with warnings.

We should make sure to deal with #22 first so that this doesn't go off for everything if an existing peer causes a warning.

Thoughts?