Support `allow-newer` in `constraint-set`

[andreasabel]

Initial discussion in a thread starting here: haskell-hvr/regex-tdfa#54 (comment)

I am trying to solve the following problem:

1. Set up a constraint-set to test for bytestring-0.12 for some newer GHCs.
2. I need allow-newer: bytestring as some of my dependencies do not allow bytestring-0.12 yet.
3. Some dependencies (e.g. xor) will fail to configure on older GHCs if they are bound to the installed bytestring.
4. However, some (e.g. xor) will also fail to build if their upper bound is loosened.

For example, looking at the following CI configuration:

installed: +all -mtl -transformers -unix -process -directory
  -- adding -bytestring here gives build failure for xor on GHC 7.6, 
  -- since `allow-newer` below allows for a too new `bytestring`
  -- not adding -bytestring here gives configuration failure for xor on GHC 7.6
constraint-set bytestring-0.12
  ghc: >= 8.2 && < 9.5
  constraints: bytestring ^>= 0.12.0.0
  tests: True
  run-tests: True
raw-project
  allow-newer: bytestring

• If I have a global allow-newer: bytestring, on GHC 7.6 xor fails to build: https://github.com/haskell-hvr/http-io-streams/actions/runs/5597341206/jobs/10235561064?pr=12#step:17:197
• If I restrict to the installed bytestring, on GHC 7.6 xor fails to configure: https://github.com/haskell-hvr/http-io-streams/actions/runs/5600301654/jobs/10242494046?pr=12#step:15:27

The principled solution would be to have a allow-newer that is scoped by a constraint-set rather than action globally via raw-project.

[andreasabel]

I have now more evidence why a raw-project allow-newer is harmful: it leaks into

1. the main build job
2. the other constraint-sets

So, adding a constraint set that requires a allow-newer is a non-orthogonal operation. It may destroy existing components. (A practical lecture on the curses of non-orthogonal operations has been given in the "Balance" PL of the ICFP 2006 programming contest.)

In the wild I experienced that adding a constraint set for text-2.1

• destroyed the constraint set of bytestring-0.12: https://github.com/haskell-hvr/microaeson/actions/runs/6034650205/job/16373452360
• and the regular build on GHC 8.2 (which does not seem to ship text yet): https://github.com/haskell-hvr/microaeson/actions/runs/6034792199/job/16373883082

-- `allow-newer: text` breaks aeson in the GHC-8.2 build
enabled: < 8.2 || > 8.3

constraint-set bytestring-0.12
  ghc: >= 8.2
  constraints: bytestring ^>= 0.12
  constraints: text < 2.1
    -- otherwise aeson fails to build
  tests: True
  run-tests: True

constraint-set text-2.1
  ghc: >= 8.2 && < 9.8
  constraints: text ^>= 2.1
  -- Cannot build aeson with text-2.1 atm (2023-08-31)
  tests: False

constraint-set text-2.1-tests
  ghc: >= 9.8
  constraints: text ^>= 2.1
  tests: True
  run-tests: True

-- For constraint-sets
raw-project
  allow-newer: bytestring
  allow-newer: text

constraints: text < 2.1 had to be added to the constraint-set bytestring-0.12 after adding the constraint-set for text. Also GHC-8.2 had to be disabled with enabled: < 8.2 || > 8.3.

This strengthens the case for the OP, a local allow-newer.