scalacenter/bloop

Tracking flaky tests

ckipp01 opened this issue · 1 comments

Related to the #1680 epic I wanted to create an issue to start actually tracking some of the tests that I'm continually seeing failing. As I notice them I'll start listing them in here and then will try to address them one the build has been cleaned up fully.

  • bloop.scalajs.ScalaJsToolchainSpec seems to be failing often in the platform bridges
  • bloop.dap.DebugServerSpec seems to be failing pretty often.
  • bloop.launcher.LatestMainLauncherSpec also seems to be failing often. The weird thing is that it seems to be failing on the check if python is there. There is a timeout of 2 second, so maybe we try to bump that up, but that seems odd to be failing.
  • bloop.dap.DebugProtocolSpec

referring to the python check up above

def isPythonInClasspath: Boolean = {
if (!detectPython) false
else runCommand(List("python", "--help"), Environment.cwd, Some(2)).isOk
}
}

I think the problem with the tests is that they assume strict timings, so it's easy for the to be flaky 🤔

I've been trying to fix them, but I haven't been able to make too much of a dent here.