StanfordAHA/garnet

Fix/disable broken CI tests

steveri opened this issue · 1 comments

steveri commented

Garnet CI has not worked for a Very Long Time, and we have been routinely merging changes even though many/most of the CI tests consistently fail. To fix this, we propose:

  • remove outdated/failing buildkite test garnet-global-buffer-plus-fabric-test
  • prune away failing pytests unit tests until both github action test and buildkite/checkin-gf tests pass

After the smoke clears, we should have four working tests for each pull:

  • github action test (pytest unit tests)
  • buildkite/garnet (full docker build plus "daily"/"pr" regressions on gf machine e.g. r7cad-docker.
  • buildkite/checkin-gf pytest unit tests except buildkite instead of github action)
  • buildkite/mflowgen (amber build, up through floorplanning, for PE, mem, and GB tiles).

Disable garnet-global-buffer-plus-fabric-test

Disable/skip failing pytest unit tests

For each failing test, I plan to implement the pytest.skip mechanism along with a message something like

pytest.skip("(Temporarily?) skip failing test as part of the spVspV-to-master merge, see garnet issue 924")
steveri commented

This issue is addressed by pull #952 . All tests pass as of that pull.

In the new regime, three tests run on every garnet pull and check-in:

  • build (github action): 26 pytests
  • buildkite/garnet (buildkite): aha "pr" regressions, same as above (?)
  • buildkite/mflowgen: RTL build and amber PD up through floorplanning

In addition, any pull to garnet master results in a fourth test

  • StanfordAHA Flow (aha "pr" regressions via buildkite triggered by github action)

This fourth test is completely redundant with buildkite/garnet, and is triggered by a mechanism that automatically runs regressions after a pull to the master of any repo used by aha, sort of. And garnet is a repo used by aha. So. That's why it runs.