Have vcpkg-sys to embbed vcpkg itself in vcpkg-rs
fungos opened this issue · 11 comments
vcpkg could make use of a vcpkg-sys crate that will use cc crate to compile vcpkg itself. And by having vcpkg project and ports as a git submodule, this would enable vcpkg-sys to provide all the files necessary (except CMake and MSVC build tools obviously) to build C and C++ native libraries to use as Rust dependencies.
Creating a cargo-vcpkg plugin to download and build vcpkg packages would improve drastically the user experience on Windows when using MSVC toolchain.
Imagine:
c:\rust\relm> cargo vcpkg gtk
c:\rust\relm> cargo run --example button
Better yet if somewhat the first step could be done via a build.rs script in gtk-rs itself.
By having vcpkg embedded, all native libraries may end in the project target folder or in .cargo folder and be dependent of toolchain version.
Great ideas. I'd love it if vcpkg ended up being able to do something like that.
An issue is that all libraries that are used by a build should come from the same vcpkg tree so that they are self consistent. I think this means either using environment variables, or only allowing the top-level binary being built to choose which vcpkg instance to use.
For my own use, I have the specific version of vcpkg that I require pinned as a submodule in the source tree for the top level binary. Enternal to cargo, I have to arrange to run booststrap-vcpkg.bat, vcpkg install the right libraries and then set some environment variables before running cargo.
I don't think the build.rs from the top level binary crate is useful for setting up libraries for crates that it depends on because it runs so late in the build process.
It seems like with some work a workflow like this should be possible, automatically creating a private vcpkg installation inside target/:
VCPKGRS_REV=https://github.com/Microsoft/vcpkg?branch=whatever
VCPKGRS_DYNAMIC=1
cargo build
Does that look reasonable to you?
That said, I like to know where my vcpkg installation is because it takes many hours to rebuild all of my dependencies...
(sorry - I hit the button before I had finished my comment)
Yes, that looks like a good solution.
Was there any progress since? In fact with Microsoft being more interested in Rust recently, maybe they will be keen to improve the support?
cc @rylev @adrianwithah (summoning the authors of COM stack in Rust https://github.com/microsoft/com-rs)
Sorry, this is not something I have had time to work on. If anyone wants to take a look at it, that would be great.
Recently in another project I needed and solved something similar (something using curl on Windows), then I took the time to adapt that solution to Rust and it works amazingly well. It is still a proof-of-concept, and I truly believe this is the way forward, hereby I present AutoVcpkg!
I'm still unsure as how to make this go forward, there are a few things to work so it will be smooth as butter for anything, but I'm really happy with current results. This can simply open the door to a lot of (easily usable) native dependencies in Rust.
Hello!
I'm now making it use vcpkg-rs internally, it works but I have some issues:
- vcpkg::find_package functions do print the cargo_metadata at the end, it would be nice to have an alternative way where it doesn't;
- I can't pass an alternative vcpkg root, this must be done via env vars. Would be nice to have it as an option in the Config builder;
- I would like to have access to the raw libs names used in the metadata (not the full filename, only eg. "curl", "crypto", "ssl" used to build the rustc-link line);
And, not related to vcpkg-rs itself, but in the case of curl, I was having a lot of trouble because the linker expects the libs to be correctly ordered and the order coming from vcpkg-rs isn't correct.
- vcpkg gives: curl,crypto,ssl,z
- linker needs: curl,ssl,crypto,z
If crypto comes before curl or ssl we get a lot of linker errors and unfortunately cargo does not have a way to pass a lib group -Wl,--start-group,-lcurl,-lcrypto,-lssl,-lz,-Wl,--end-group which would solve this. My current hack is to force crypto again as a last line, but this obviously do not scale. Do you have any ideas?
(I only have a couple of minutes right now but I wanted to get back to you so this is a bit terse.)
Did you try creating a vcpkg::Config and calling cargo_metadata(false) on it? That is intended to do what you are asking.
The cargo metadata lines that would be emitted if cargo_metadata is enabled are in a vec in Library even if printing is disabled. You could parse those lines to see the library names if you really want them. Alternatively and probably better would be to return the short names in the Library struct. (PR welcome if you want to do it.) Same for passing the vcpkg root in using the config. I suppose the value in the config should supercede the env var.
Regarding the link order, are you using Linux? I had an issue where I could not tell the link order from what is read from the vcpkg tree and had to println!() an extra cargo:rustc-link-lib=foo to make it work. I'm not sure if vcpkg itself on a single pass ld platform will do it correctly - it's something that could use further investigation.
The order of dependency of the packages themselves is in Library::ports, but that won't help crypto,ssl of course.
PR #16 done, not sure about passing &Config there, please take a look.
Yes, the linking issue happens on Linux (and probably OSX too), unfortunately it is a common problem with C and C++ linking with ld. Cargo could do a better job supporting groups there.
My solution was exactly like yours, but that will make us run after ourselves for each library or let possible users deal with it. For now, this looks like:
// vcpkg-rs gives: curl,crypto,ssl
// curl needs: curl,ssl,crypto
if cfg.packages.contains(&"curl".to_string()) {
#[cfg(not(target_os = "windows"))]
println!("cargo:rustc-link-lib=crypto");
}
But I'll try to use the short lib names from the PR too, so if "ssl" and "crypto" are present, put an extra crypto last.
Another solution I'm evaluating is to make CMake print the right order by abusing target properties inside autovcpkg shim-sys hack.
@fungos I'm not sure if I told you when I cut the crate but that PR is there as of 0.2.8 from December.
If you are interested - I just made a new tool cargo-vcpkg like you suggested. It reads metadata from the root crate and anything it depends on to find which packages are required and then installs them. The top-level crate specifies a git url and a rev/branch/tag to check out. cargo vcpkg build downloads vcpkg, builds it and installs the packages in one step. It still requires the VCPKGRS_DYNAMIC environment variable (or RUSTFLAGS for static) to be set on Windows (no VCPKG_ROOT though). I may even be able to do away with that at some point.
Hi, this is great news, nice to see it going forward.
For myself, my use case was mainly gtk-rs, but after trying to fix their issues with pixmaps and meson I decided it wasn't worth the effort and abandoned it for something else, but this is certainly an awesome step forward.