Clash slow on Windows using GHC 9.8
Opened this issue · 3 comments
christiaanb commented
Observed on the CI runs of #2768, I see that running the vector testsuite takes:
martijnbastiaan commented
martijnbastiaan commented
This could be the same as what doctest-parallel is experiencing. I've collected some results.
A few observations observations:
- Performance has gradually been declining on Windows
- It got very bad with 9.6
I don't think there is a point in comparing the timings between OSes directly. I don't know what hardware is picked for each OS.
Windows
| GHC | doctests | spectests | example project |
|---|---|---|---|
| 8.4.4 | 4 | 65 | 30 |
| 8.6.5 | 3 | 66 | 29 |
| 8.8.4 | |||
| 8.10.7 | 3 | 83 | 40 |
| 9.0.2 | 4 | 78 | 30 |
| 9.2.8 | 4 | 85 | 30 |
| 9.4.7 | 4 | 101 | 26 |
| 9.6.3 | 14 | 351 | 30 |
| 9.8.1 | 14 | 338 | 31 |
doctest-parallel is broken on 8.8.4 on Windows.
Linux
| GHC | doctests | spectests | example project |
|---|---|---|---|
| 8.4.4 | 0 | 20 | 19 |
| 8.6.5 | 0 | 26 | 19 |
| 8.8.4 | 1 | 27 | 20 |
| 8.10.7 | 1 | 27 | 20 |
| 9.0.2 | 1 | 29 | 19 |
| 9.2.8 | 0 | 27 | 18 |
| 9.4.7 | 0 | 27 | 19 |
| 9.6.3 | 1 | 27 | 14 |
| 9.8.1 | 1 | 24 | 21 |
macOS
| GHC | doctests | spectests | example project |
|---|---|---|---|
| 8.4.4 | |||
| 8.6.5 | |||
| 8.8.4 | |||
| 8.10.7 | |||
| 9.0.2 | |||
| 9.2.8 | 2 | 44 | 19 |
| 9.4.7 | 1 | 41 | 20 |
| 9.6.3 | 1 | 31 | 19 |
| 9.8.1 | 2 | 30 | 21 |
doctest-parallel's CI is broken on 8.4.4-9.0.2 on macOS.
martijnbastiaan commented
I've spun up a bunch of CI jobs to test cabal run doctest on doctest-parallel. It seems that a patch introduced in 9.8 but backported to 9.6.3 and 9.4.8 causes issues. Note that the 9.4 job you linked runs on 9.4.7, which doesn't have that patch yet.
Results
| os | ghc | mean |
|---|---|---|
| windows-latest | 9.10.1 | 14.37 |
| windows-latest | 9.8.2 | 15.82 |
| windows-latest | 9.8.1 | 13.63 |
| windows-latest | 9.6.6 | 14.80 |
| windows-latest | 9.6.5 | 14.18 |
| windows-latest | 9.6.4 | 13.75 |
| windows-latest | 9.6.3 | 13.61 |
| windows-latest | 9.6.2 | 4.48 |
| windows-latest | 9.6.1 | 4.88 |
| windows-latest | 9.4.8 | 14.08 |
| windows-latest | 9.4.7 | 4.79 |
| windows-latest | 9.4.6 | 4.39 |
| windows-latest | 9.4.5 | 4.25 |
| windows-latest | 9.4.4 | 4.50 |
| windows-latest | 9.4.3 | 4.53 |
| windows-latest | 9.4.2 | 4.53 |
| windows-latest | 9.4.1 | 4.89 |
| windows-latest | 9.2.8 | 4.16 |
| windows-latest | 9.2.7 | 3.82 |
| windows-latest | 9.2.6 | 3.62 |
| windows-latest | 9.2.5 | 3.42 |
| windows-latest | 9.2.4 | 3.60 |
| windows-latest | 9.2.3 | 3.66 |
| windows-latest | 9.2.2 | 3.58 |
| windows-latest | 9.2.1 | 3.65 |
| windows-latest | 8.10.7 | 3.58 |
| windows-latest | 8.10.6 | 3.63 |
| windows-latest | 8.10.5 | 3.53 |
| windows-latest | 8.10.4 | 3.88 |
| windows-latest | 8.10.3 | 3.75 |
| ubuntu-24.04 | 9.10.1 | 0.89 |
| ubuntu-24.04 | 9.8.2 | 0.85 |
| ubuntu-24.04 | 9.8.1 | 0.90 |
| ubuntu-24.04 | 9.6.6 | 0.89 |
| ubuntu-24.04 | 9.6.5 | 0.86 |
| ubuntu-24.04 | 9.6.4 | 0.90 |
| ubuntu-24.04 | 9.6.3 | 0.91 |
| ubuntu-24.04 | 9.6.2 | 0.92 |
| ubuntu-24.04 | 9.6.1 | 0.90 |
| ubuntu-24.04 | 9.4.8 | 0.88 |
| ubuntu-24.04 | 9.4.7 | 0.92 |
| ubuntu-24.04 | 9.4.6 | 0.92 |
| ubuntu-24.04 | 9.4.5 | 0.88 |
| ubuntu-24.04 | 9.4.4 | 0.87 |
| ubuntu-24.04 | 9.4.3 | 0.87 |
| ubuntu-24.04 | 9.4.2 | 0.86 |
| ubuntu-24.04 | 9.4.1 | 0.86 |
| ubuntu-24.04 | 9.2.8 | 0.99 |
| ubuntu-24.04 | 9.2.7 | 0.89 |
| ubuntu-24.04 | 9.2.6 | 0.94 |
| ubuntu-24.04 | 9.2.5 | 0.89 |
| ubuntu-24.04 | 9.2.4 | 0.90 |
| ubuntu-24.04 | 9.2.3 | 0.89 |
| ubuntu-24.04 | 9.2.2 | 0.90 |
| ubuntu-24.04 | 9.2.1 | 0.88 |
| ubuntu-24.04 | 8.10.7 | 0.95 |
| ubuntu-24.04 | 8.10.6 | 0.93 |
| ubuntu-24.04 | 8.10.5 | 0.94 |
| ubuntu-24.04 | 8.10.4 | 0.94 |
| ubuntu-24.04 | 8.10.3 | 0.95 |
| ubuntu-24.04 | 8.10.2 | 0.94 |
| ubuntu-24.04 | 8.10.1 | 1.06 |