Prefix the vendored libraries

[clecat]

In a recent project, I met various issues due to the forked libraries vendored by dream.
Indeed, it was really hard to have the latest versions of many other dependencies due to conflicts between their own dependencies and the dependencies of dream.

Indeed, when a library is vendored by dream but is also required by another dependency, opam is unable to understand which one of them he should use.

I have temporarily solved this issue by forking dream and all it's vendored libraries, and prefixing everyone of them following the format dream-*. This is not optimal, but it is the only option that I could think about.

Could it be also done on the main repository, so the problem would be permanently solved ?

[aantron]

How invasive is the change? Can you show the patch, either pasting it here as a diff, or opening an exploratory PR with your change, on this repo?

We could do this in Dream. If the patch is very invasive, it could slow down Dream development, since sometimes Dream needs to push something upstream to those vendored libraries (i.e., they are used as development forks). If we have to translate too many names for upstream patches, it might be a drag. But it might still be worth it. So please show the patch :)

[dinosaure]

I would like to say that for an opam perspective and dune, the situation where httpaf.cm{x}a is available into the OPAM switch and available into dream become a fatal error where dune is not able to choose the right one. Add a prefix name on vendored libraries allows a global sane workflow.

[clecat]

My apologies, I messed up with mailing rules and didnt see the notification.

The modifications that have to be done in the sub-libraries themselves are the following:

• Public name in dune file from foo to dream-foo
• Change the name of the foo.ml module for dream_foo.ml
• Rename foo.opam for dream-foo.opam
• Rename the project name in the dune-project (not sure if it is necessary, but might be cleaner)

In dream of in the sub-libraries that are depending on others (e.g. gluten-async needs gluten; h2 needs httpaf), you will need to:

• Modify the name of the dependencies in the dune files.
• Modify the name of the dependencies in the opam file as well.
• Rename every open Foo to open Dream_foo and every access to a function of the Foo library with Dream_foo

In dream only, update the src/vendor/dune file, by changing every name, public_name, modules and dependencies to their updated name.

Here is the diff for dream, I might have changed too much things, but the idea is here:
dream-diff.txt
And here for h2:
h2-diff.txt
And for gluten (most likely the smaller one):
gluten-diff.txt

[aantron]

@clecat, thanks for the diffs and for uploading your prefixed branches. I was able to apply most of the changes in branch prefix-vendored in this repo. I opened #200 to continue the work. We need to rename dream-httpaf to dream-httpaf_ or similar. Since you already have the workflow figured out, could you resume that? I will then pull in your additional changes into the PR, do any final touch-ups, and merge it. See #200 for slightly more detail.

[clecat]

Thanks a lot, I will try to work on it as soon as possible, however i wont be able to do it before the end of the week, but I'll keep it in mind

[dinosaure]

Thanks, @aantron, same as @clecat, I will not be available for a moment but your PR will help us a lot!