Enhancement request: interface to get the wasi-libc version at runtime

Question

Enhancement request: interface to get the wasi-libc version at runtime

Opened this issue 7 months ago · 22 comments

I work on a Swift testing library, swift-testing (https://github.com/apple/swift-testing). I'm looking at adding Wasm/WASI support using the SwiftWasm toolchain (https://www.swiftwasm.org/). One of the things we do when starting a test run is log some diagnostic info including the OS version so that developers can say "oh, this test always fails on Ubuntu 23 (or macOS 30 or Windows 12 etc.), I'd better fix that."

For WASI, I'm told that the platform has a concept of a "current version" and I see that wasi-libc internally knows its own version, but there's no interface to read it at runtime. So I'm hoping such an interface could be added. I don’t have a particular interface design in mind.

Thanks!

Answer 1 · 2024-05-02T21:30:37.000Z

How about this? Much like Emscripten does, we generate a version.h file containing #define WASI_LIBC_VERSION "<version string>". I'm just not really sure what version you are seeing when you say "wasi-libc internally knows its own version" (MUSL's version.c is filtered out in the Makefile) and I'm not exactly sure what version we would insert here anyways: the wasi-sdk-* tag? Does that make sense?

Answer 2 · 2024-05-02T23:59:49.000Z

That would work for my needs, yeah. I don't have any constraints on the format of the string, but having something here would allow us to better-triage bug reports when devs are using WASI.

Are you saying the header is already available, or would need to be added?

Answer 3 · 2024-08-30T18:06:03.000Z

Thinking about this more: the wasi-sdk-* tag would be perfect for our needs as it corresponds most closely to an OS release in traditional environments. :)

Answer 4 · 2024-09-02T06:21:39.000Z

i don't understand why such an api is necessary at all.
if someone linked wasi-libc to their app, he knows which version it is.

Answer 5 · 2024-09-03T16:17:10.000Z

@yamt That's a fair question. I work on a library that's a component of the Swift toolchain, and that library doesn't typically know what version of wasi-libc (or other parts of the WASI stack) it's linked against. We know we're building for WASI and for Wasm32/Wasm64, but do not have version information.

When we get a bug report, we unfortunately can't always rely on the human-authored parts of the report being detailed enough for us to figure out what WASI bits are being used. We want to be able to include version information for the environment in the auto-generated portion of the bug report.

I hope that clarifies things!

Answer 6 · 2024-09-03T16:58:55.000Z

@grynspan, thanks for the clarification. One more question: we added a VERSION file to wasi-sdk a while ago (PR) and it probably has all the information you need. Is that enough? I don't know exactly how your users are doing things, but might it be easier for the Swift toolchain to emit that information as well as its own version information so that users can give you everything together? I guess what I'm asking is: do you (or can you) ask users to gather toolchain information statically (meaning we don't need a runtime-accessible API for this) or is there some other thing I don't understand here that makes this a runtime requirement (e.g., you're printing stack traces that end with "here's what toolchain you're using")? Maybe an example of how Swift users typically report bugs to your library would help.

Answer 7 · 2024-09-03T17:07:01.000Z

I guess there is nothing POSIX/musl that works like https://man7.org/linux/man-pages/man3/gnu_get_libc_version.3.html?

Perhaps we could fit it into the output of uname? Or is that too hacky?

Answer 8 · 2024-09-03T17:10:25.000Z

@abrown Yes, I think that value (specifically the line indicating 21.11gdb977fd767ca from the PR) would be plenty for us.

The expectation is that when you run swift test, one of the lines of output we write to stderr or stdout is:

OS Version: $VERSION

Where $VERSION is "the version of the OS." For WASI, that would presumably be 21.11gdb977fd767ca. We need to gather this value at runtime because we're emitting it at runtime into the output stream. Swift Testing has at least 3 build workflows (as an Xcode project, as a Swift package, and as a toolchain component using CMake), only one of which has any connection to building the toolchain, so a compile-time value derived from the toolchain build is likely unavailable. Moreover, the SwiftWasm project (AFAIK) may be built with one version of the WASI SDK, but then downstream components that link to the SwiftWasm toolchain's runtime and test library might get built with a different version (I don't know if that's supported/possible, I'm just saying if it is possible, we likely need to support it.)

So having an exported "version" symbol is ideal for us because we can basically just say "Hey, WASI, what version are you?" and then report it without needing to plumb it through layers of Swift that may or may not even exist.

@sbc100 We actually use the C function uname() on Linux already, so if you wanted to plumb this string through there, we'd get it "for free" which would be lovely. But I don't want to prescribe any particular technical solution if it makes the solution harder to implement on the WASI side.

Answer 9 · 2024-09-03T17:16:19.000Z

@sbc100 We actually use the C function uname() on Linux already

If thats the case then is the thing you really want to know the the OS/host version, and not the libc version? i.e. does your existing code have any way to detect the libc version for glibc/bionic/musl at runtime?

If you are after the name / version of the host environment I doubt that is something that WASI will add.

Answer 10 · 2024-09-03T17:18:23.000Z

Re-reading your reply it look like you are indeed wanting the OS version. In that case I don't think that version of wasi-libc is important/relevant here and WASI is unlikely to give you any kind of host version information, so I think just using whatever info the uname() gives your best bet.

Answer 11 · 2024-09-03T17:18:51.000Z

If thats the case then is the thing you really want to know the the OS/host version, and not the libc version? i.e. does your existing code have any way to detect the libc version for glibc/bionic/musl at runtime?

What I'm after is, ideally, I think, the version of wasi-sdk, but the version of wasi-libc will suffice for this diagnostic purpose because we ought to be able to tie it back to a specific WASI SDK release manually. I think I conflated the two when I filed this issue, which is my own fault for not knowing enough about WASI as I ought to. Sorry about that.

I'm not terribly interested in information about the environment hosting WASI, as it shouldn't matter in theory (although if you did plumb it through too, I wouldn't object.)

uname() in wasi-libc does not provide any information today.

Edit: To clarify "the version of the OS": in the context of SwiftWasm, WASI is the OS, not the host.

Answer 12 · 2024-09-03T17:25:01.000Z

I think it could make sense add the version of WASI we are using to the uname()? e.g. wasip1 vs wasip2.

But I don't think the version of wasi-sdk/wasi-libc makes sense there TBH. I also don't think you should think of the wasi-sdk version as your OS version.. maybe the version of underlying WASI API being used is what you want? (e.g. wasip1 vs wasip2).

Answer 13 · 2024-09-03T17:27:39.000Z

I would have to defer to you here—if that's the appropriate string to expose, then we'll take it.

Answer 14 · 2024-09-04T00:39:21.000Z

@sbc100, the wasip1/wasip2 string seems valuable-ish but I think they're asking for more: i.e., to reproduce a bug they're going to need all of that toolchain information. What do we think of my original idea up here to add a version.h file that the Swift toolchain could use to retrieve some version definitions?

Answer 15 · 2024-09-04T00:44:11.000Z

@sbc100, the wasip1/wasip2 string seems valuable-ish but I think they're asking for more: i.e., to reproduce a bug they're going to need all of that toolchain information. What do we think of my original idea up here to add a version.h file that the Swift toolchain could use to retrieve some version definitions?

I'm not sure that is true. If we are looking for an analog of uname I don't exposing the version of wasi-libc or wasi-sdk is that right way to go. IIRC they are currently not reporting the version of glibc on linux, so I don't see why we want report out libc version either.

As I said, far more useful would be the name and version of the runtime.... but that seems unlikely. So the version of the WASI interface seems like the best compromise here.

Answer 16 · 2024-09-04T00:50:19.000Z

@sbc100, the wasip1/wasip2 string seems valuable-ish but I think they're asking for more: i.e., to reproduce a bug they're going to need all of that toolchain information. What do we think of my original idea up here to add a version.h file that the Swift toolchain could use to retrieve some version definitions?

Sorry, I think I misread.

Don't we already include version.h? If not then I agree adding that sounds very reasonable.

What I'm not so sure about is adding a wasi_libc_get_version() runtime interface. We could add such a thing for sure, and it would work much like glibc's gnu_get_libc_version, but I would not encourage folks to use such an API as a surrogate for uname (i.e. the OS version). Instead I think we should do our best to provide a maximally useful uname.

Answer 17 · 2024-09-04T01:38:56.000Z

Yeah, I agree with your reticence to add the special function stuff. @grynspan, if we give you a version.h to include that has all the relevant things you need, will you be able to plumb that through to where you need it?

Answer 18 · 2024-09-04T01:52:21.000Z

At the risk of going off topic, I assume there is no appetite for providing something like uname or User-Agent that might allow client code to know on what platform they are running on for the purposes of debugging / bug reporting? Perhaps there is some way we could allow such information to be gathered/aggregated without actually exposing it the information to the user space code?

Answer 19 · 2024-09-04T03:17:02.000Z

You mean in WASI?

Answer 20 · 2024-09-04T04:00:15.000Z

I guess there is nothing POSIX/musl that works like https://man7.org/linux/man-pages/man3/gnu_get_libc_version.3.html?

iirc musl intentionally avoids having that kind of features.
see the second item in https://wiki.musl-libc.org/faq.html

Answer 21 · 2024-09-04T14:30:07.000Z

IIRC they are currently not reporting the version of glibc on linux, so I don't see why we want report out libc version either.

Correct—I had mistakenly conflated the libc version with the wasi-sdk version when filing this issue.

Yeah, I agree with your reticence to add the special function stuff. @grynspan, if we give you a version.h to include that has all the relevant things you need, will you be able to plumb that through to where you need it?

That would suite us perfectly fine, yeah. Honestly, any of the proposals in this issue would be sufficient for us. :)

Answer 22 · 2024-09-05T18:05:14.000Z

Let's see what everyone thinks of: WebAssembly/wasi-sdk#487.