oughtinc/ice

Tests don't run on fork PRs

alexmojaki opened this issue · 0 comments

In PRs such as #208 (cc @smithjessk) from external contributors, the Tests check gets stuck on yellow saying:

Expected — Waiting for status to be reported

Screenshot from 2023-01-12 13-02-27

Looking at https://github.com/oughtinc/ice/actions/workflows/tests.yml, the action isn't running at all, so I take it on: [push] doesn't apply to forks.

Based on https://github.com/orgs/community/discussions/26698, it seems the check is 'Expected' because of the main branch protection rule, and of course it remains expected forever since it never runs.

If we do run tests on fork PRs, then currently some tests will fail as secrets such as OPENAI_API_KEY won't be accessible. #193 will partly solve this by running primer recipes in test mode. A few non-primer recipes call openai_complete directly, but they will probably be moved out of the repo soonish anyway.