warpforge fails if `rio` is a symlink
Closed this issue · 5 comments
I'm currently packaging warpforge for Nix and am compiling rio as a separate package.
In the Nix ecosystem, it's idiomatic to use symlinks for interop of packages, so my go-to solution of packaging warpforge would mean that rio is symlinked into WARPFORGE_PATH, instead of copied.
However, trying to execute a plot results in a ton of repeated error messages, essentially boiling down to:
the invoke runc for rio pack of output failed engine reported error: warpforge-error-executor-failed: executor engine failed: the runc engine reported error: time="2023-02-02T15:53:09+01:00" level=error msg="runc run failed: unable to start container process: exec: \"/.warpforge.container/bin/rio\": stat /.warpforge.container/bin/rio: no such file or directory"
Digging into the code a bit, it seems like the plugin directory is bindmounted into the runc containers, and it seems like mounted symlinks are a general pain point of runc-based containers, as indicated by e.g. this stackoverflow question.
I know that this is not the intended/official way of installation, but it would be great if symlinked plugins could still be supported, or if warpforge status and/or warpforge health could complain about symlinks otherwise.
Yeah we would have to follow the link then copy the actual binary somewhere we can mount it. We can add some more checks for file mode bits in the health check though pretty easily I think. On an interesting note, runc isn't mounted and so it can be a symlink.
Hm, yeah, so far we've only aimed for the flexibility of having a configurable WARPFORGE_PATH for plugins, but not further than that.
I'm generally a (huge) fan of symlink farming, so I readily understand the desire for this. :)
We've already sprouted some code for doing symlink resolves for similar mounting purposes but focused on TLS certificates files. That sort of logic should be able to be applied here too, where we want to mount plugin binaries into the contained environment. I think writing a PR for that should be something fairly approachable, even?
The current symlink resolve code doesn't properly handle mounting relative symlinks where the symlink will be mounted to a different path than the source path. It's also difficult to write a unit test for it to make sure it handles relative/absolute paths correctly because os.DirFS doesn't provide an Lstat. We'd have to duplicate the DirFS implementation and provide a Stat which doesn't follow symbolic links.
@feathecutie Check out the above branch and let me know if that works.
This works perfectly, thank you!