brain-score/vision

Review jenkins after OM downtime

Closed this issue · 0 comments

Task: Review Jenkins/OM test failures happening during or after OM downtime.

Findings: Of the PRs that have been run since downtime started, multiple have failed tests.

  • For unittest_plugins, all failures were specific to the PR; all but one were appropriate failures; one ran out of memory because the submission contained too many models (tracked in separate issue, should not occur for all plugin submissions).
  • For unittest_brainscore, test runs are timing out for any PR that contains non-plugin-related code changes; not a blocker for submission, but is a blocker for dev. progress