open-mpi/hwloc

Consider exposing a function to set an object's subtype or the malloc() that hwloc uses

HadrienG2 opened this issue · 3 comments

Overally, hwloc is remarkably usable in those unfortunate situations where you cannot guarantee that the application will end up linking against the same libc as the hwloc build. Like, say, when linking against the official hwloc DLL on Windows, which has taken libc/CRT proliferation to a ridiculous extreme.

As far as I'm aware, no hwloc function expects the user to malloc() a buffer, pass it to hwloc, and let hwloc free() it at the end of its lifetime. Instead, most hwloc functions copy the user input into a new hwloc-managed allocation, which trivially ensures malloc/free consistency. And for the few that do acquire ownership of user-provided objects, they normally enforce that objects must have been previously allocated using an hwloc-provided function, which again ensures malloc/free consistency. So usually, the only thing you lose in a multi-libc scenario is that you cannot read the errno values set by hwloc anymore (since your errno is in a different castle libc).

There is one exception, however, which is that users are explicitly encouraged by documentation to set the subtype field of objects to a dynamically allocated string. Which will be allocated by the libc that the application links against, and later freed by the libc that hwloc links against. This pattern is thus not safe in multi-libc scenarios.

I can think of two ways to resolve this:

  1. The way that would be most consistent with the API design of existing hwloc functions would be to expose an hwloc function to set an object's subtype, and stop recommending that users set the subtype field directly in the docs.
  2. Alternatively, a more flexible approach would be to expose the malloc implementation that hwloc uses. hwloc_alloc() almost does this, but inconviently requires access to the parent topology, and is a little bit wasteful by requiring page-aligned memory which is not necessary in this particular case.

Oh that's interesting, thanks. I think I'd vote for (1) a hwloc_obj_set_subtype() but I need to take a deeper look at the potentiel implications of (2) before deciding.
Do you have a strong need for a fix in a stable release or is it ok if this change only goes in future 2.11 and 3.0 in a couple months?

This is not urgent, take as much time as you need :)

I am posting 2.11rc1 right now with this change.