LIBCx - kLIBC Extension Library

The kLIBC Extension Library extends the functionality of the kLIBC library by adding a number of high demand features required by modern applications. The kLIBC library is not actively maintained any more and extending it with the needed functionality is not always a trivial task. LIBCx is a good place to deploy such extensions as it does not require touching complex kLIBC internals and therefore cannot introduce new bugs and regressions in a sensitive piece of software the base C runtime library naturally is.

It is no doubt that all the functionality provided by LIBCx logically belongs to the C runtime and should eventually migrate to kLIBC (or to its probable successor) and this is the final goal of this project. Until then, applications should be manually linked with LIBCx (in addition to the implicit linking with kLIBC performed by the GCC compiler under the hood) in order to use all the implemented extensions.

Currently, LIBCx provides the following extensions:

Improved advisory file locking using the fcntl() API. The implementation provided by kLIBC uses DosSetFileLocks and is broken as it does not guarantee atomicity of lock/unlock operations in many cases (like overlapping lock regions etc.) and does not have deadlock detection.
Improved positional read/write operations provided by pread() and pwrite() APIs that guarantee atomic behavior. kLIBC emulates these functions using a pair of lseek and read()/write() calls in non-atomic manner which leads to data corruption when accessing the same file from multiple threads or processes.
Improved select() that now supports regular file descriptors instead of returning EINVAL (22) on them as kLIBC does. Regular files are always reported ready for writing/reading/exceptions (as per POSIX requirements).
Implementation of poll() using select(). kLIBC does not provide the poll() call at all.
Implementation of POSIX memory mapped files via the mmap() API (declared in sys/mman.h).
Automatic installation of the FPU exception handler on the main thread of the executable (prior to calling main()) as well as on any additional thread created with _beginthread() (prior to calling the thread function). This exception handler automatically recovers from infamous crashes in programs using floating point math caused by various bogus Gpi and Win APIs that change the FPU control word and do not restore it upon return.
Improved read(), __read(), _stream_read(), fread() and DosRead() calls with workarounds for the OS/2 DosRead bug that can cause it to return a weird error code resulting in EINVAL (22) in applications (see #21 for more information) and for another DosRead bug that can lead to system freezes when reading big files on the JFS file system (see #36 for more information).
Improved flushing of open streams at program termination. In particular, buffered streams bound to TCP/IP sockets are now properly flushed so that no data loss occurs on the receiving end. TCP/IP sockets are used instead of pipes in many OS/2 ports of Unix software for piping child process output to the parent (due to the limitation of kLIBC select() that doesn't support OS/2 native pipes).
New exeinfo API that allows to examine an executable or a DLL without actually loading it for execution by the OS/2 kernel.
New spawn2() API built on top of spawnvpe() that provides enhanced functionality such as standard I/O redirection and thread safety.
Placing the regular C heap in high memory (when it's availalbe) by default. This behavior may be controlled by the LIBCX_HIGHMEM environment variable, see the notes below.
Implementation of the getaddrinfo and getnameinfo family APIs that provide a modern, reentrant and address family neutral way to resolve IP addesses to host names and vice versa. On OS/2 these APIs only support the IPv4 address family. Note that reentrancy is guaranteed by a kLIBC mutex so that only one thread can call these functions at a time. Applications can get access to this API by including <libcx/net.h>. They will be moved to the standard header <netdb.h> later, once LIBCx is integrated with kLIBC.
Implementation of the if_nameindex and if_nametointex family APIs that provide a modern, address family neutral way to map interface names to 1-based indexes and vice versa. On OS/2 these APIs only support the IPv4 address family. Applications can get access to these APIs by including <libcx/net.h>. They will be moved to the standard header <net/if.h> later, once LIBCx is integrated with kLIBC.
Implementation of the getifaddrs family API that provides a modern and address family neutral way to query all network interfaces. On OS/2 this API only supports the IPv4 address family. Applications can get access to this API by including <ifaddrs.h>.
New shmem API to conveniently allocate shared memory and use it between LIBCx processes. As opposed to system-provided DosAllocSharedMem API, this new API allows to track usage of each shared memory block via opaque "handles" of a special SHMEM type, similar to handles to open files. This approach helps in managing complex ownership of shared memory between multiple processes to ensure its proper release and avoid memory leaks. Refer to <libcx/shmem.h> for a complete documentation.
New handles API to transfer various types of handles between LIBCx processes. Currenlty supported are SHMEM handles and socket file descriptors. Note that normal file descriptors are not yet supported. There are two transfer modes: send (where the transfer is initiated by the sender) and take (where the transfer is initiated by the receiver). Refer to <libcx/handles.h> for a complete documentation.
EXPERIMENTAL. Automatic setting of the Unix user ID at process startup to an ID of a user specified with the LOGNAME or USER environment variable if a match in the passwd database is found for it. This has a numbef of side effects, e.g. all files and directories created by kLIBC functions will have an UID and GID of the specified user rathar than root. Also Unix programs will see the correct user via getuid() and other APIs which in particular will make some tools (e.g. yum) complain about the lack of root priveleges. For this reason this functionality is disabled by default and can be enabled by setting LIBCX_SETUID=1 in the environment.

Notes on `mmap()` usage

The mmap() implementation needs a special exception handler on every thread of the executable (including thread 1). There is no legal way to install such a handler in kLIBC other than do it manually from main() and from each thread's main function which is beyond mmap() specification (and may require quite a lot of additional code in the sources). LIBCx solves this problem by overriding some internal LIBC functions (in particular, __init_app) as well as _beginthread. If you use LIBCx mmap() in your executable, everything is fine since you will build it against LIBCx and the right functions will be picked up from the LIBCx DLL. But if you use LIBCx mmap() in a separate DLL (e.g. some 3rd party library), it will not work properly unless you also rebuild all the executables using this DLL against LIBCx to pick up the necessary overrides (see also Notes on EXCEPTQ usage).

Please note that it is not recommended to install any additional OS/2 system exception handlers in the application code if it uses mmap(). Extra care should be taken if such a handler is installed. In particular, all exceptions not explicitly handled by this handler (especially XCPT_ACCESS_VIOLATION) must be passed down to other exception handlers by returning XCPT_CONTINUE_SEARCH for them.

Both anonymous and file-bound memory mappings are supported by LIBCx. For shared mappings bound to files LIBCx implements automatic asynchronous updates of the underlying file when memory within such a mapping is modified by an application. These updates happen with a one second delay to avoid triggering expensive file write operations after each single byte change (imagine a memcpy() cycle) and still have up-to-date file contents (which is important in case of a power failure or another abnormal situation leading to an unexpected reboot or system hang). Note though that changed memory is always flushed to the underlying file when the mappig is unmapped with munmap() or when the process is terminated (even abnormally with a crash). If an immediate update of the file with current memory contents is needed prior to unmapping the respective region (for instance, to read the file in another application), use the msync() function with the MS_SYNC flag.

Shared mappings in two different and possibly unrelated processes that are bound to the same file will always access the same shared memory region (as if it were a mapping inherited by a forked child). This allows two unrelated processes instantly see each other's changes. This behavior is not documented by POSIX but many Linux and BSD systems implement it too and it is used in real life by some applications (like Samba).

Note that the MAP_FIXED flag is not currently supported by mmap() becaues there is no easy way to ask OS/2 to allocate memory at a given address. There is a subset of MAP_FIXED functionality that can be technically implemented on OS/2 and this implementation may be added later once there is an application that really needs it. See #19 for more information.

The msync() function supports two flags: MS_SYNC and MS_ASYNC. MS_SYNC causes the changes to be immediately written to the file on the current thread and MS_ASYNC simply forces the asynchronous flush operation to happen now instead of waiting for the current update delay inteval to end. The MS_INVALIDATE flag is not supported (silently ignored).

The mprotect() function overrides the kLIBC function of the same name (the only function from sys/mman.h ever implemented by kLIBC) in order to make sure that changing memory protection does not break functionality of mappings created with mmap(). Currently it has the following limitations:

It supports anonymous mappings only and will fail with EACCES if a mapping bound to a file is encountered within a requested region. Note that if the requested region does not intesect with any mapped regions at all, then the call is simply passed down to the kLIBC implementation of mprotect().
Partial mprotect of a single mapping, even if it is anonymous, is also not supported and will return EACCESS too. See #75 for more information.
It cannot change protection to PROT_NONE on shared mappings (and any shared memory regions, in fact) and will fail with EINVAL if such a region is encountered. This is a limitation of OS/2 itself that does not allow to decommit a shared page and doesn't provide any other way to mark that the page has no read and no write access. This limitation is also present in kLIBC mprotect().

The madvise() function is mostly a no-op as the OS/2 kernel doesn't accept any advices about use of memory by an application. The only implemented advice is MADV_DONTNEED which changes memory access semantics. This is advice is supported for private mappings in which case it causes the requested pages to be decommitted (and automatically committed again upon next access and initialized to zeroes for anonymous mappings or to up-to-date file contents for file mappings). Shared mappings are not supported due to limitations of the shared memory management on OS/2 (in particular, it's impossible to decommit a shared memory page or somehow unmap it from a given process or even remove the PAG_READ attribute from it which is necessary to cause a system exception upon next access).

The posix_madvise() function is also implemented but it simply returns 0 in all cases (including the POSIX_MADV_DONTNEED advice as it has different semantics, not compatible with MADV_DONTNEED).

Notes on EXCEPTQ usage

The EXEPTQ support is enabled automatically whenever an application is statically linked against the LIBCx DLL or its import library. There is no need to call any of the LIBCx functions to turn it on and there is currently no way to turn it off (other than call the SetExceptqOptions function from exceptq.dll directly). Any manual EXCEPTQ installation should be removed from the application code if it is linked against LIBCx to avoid double processing and other curious side effects.

Note that LIBCx doesn't statically link to any of the EXCEPTQ DLLs. It loads them dynamically at program startup. If exceptq.dll (or any of its dependencies) cannot be found at startup, the EXCEPTQ support provided by LIBCx will be completely disabled.

Notes on FPU exception handler usage

The FPU exception handler is enabled automatically whenever an application is statically linked against the LIBCx DLL or its import library. There is no need to call any of the LIBCx functions to turn it on and there is currently no way to turn it off. The solution provided by LIBCx is similar to what Mozilla has provided in NSPR for years but it has a number of key improvements which make it a better choice.

The first improvement is that LIBCx also overrides the _control87 LIBC function in order to memorize the last control word set by the program. This memorized control word is what gets restored after a bougs Gpi/Win API call instead of some default value. This means that if an application deliberately changes the FPU control word with _control87 (which is the only legal way in C/C++), it will be preserved by the LIBCx FPU exception handler. The Mozilla exception handler would simply always mask off all exceptions using the MCW_EM (0x3F) value which could be not what the program wanted. Note that in order to incorporate this behavior, the executable needs to be rebuilt against the LIBCx DLL or its import library so that it links to the right version of _control87.

The second improvement is even more important. The FPU uses the exception mask to decide if it should fix the erroneous condition on its own (e.g. by putting a specail value of Inf or NaN into the result) or raise a corresponding SIGFPE exception. If it decides to raise an exception, it does not perform any of its own fixes on the result and moves the instruction pointer to an instruction next to the one that caused the exception. If the FPU exception handler simply restores the FPU control word and continues execution (the Mozilla way), the value of the result may be not what the application expects, even although it doesn't crash. A simple example is divison by zero. Given float result = 1./0.;, the value of the result variable will not be set to INFINITY (as it should be) after the Mozilla FPU exception kicks in and recovers from the crash. This may confuse programs expecting the documented behavior. The LIBCx FPU exception handler recover from this more correctly by retrying the previous instruction (the one that generated the exception) after restoring the FPU control word. If the restored control word masks off the respective exception, it will cause the FPU to properly fill up the result when executing the failed instruction again.

Notes on `exeinfo` API usage.

The key difference of the exeinfo API from the "traditional" OS/2 DosGetResource API is that it doesn't require to load the executable file with DosLoadModule first in order to read its resources — instead, file contents is read and parsed directly by LIBCx. Besides saving some resources that would otherwise be alocated by OS/2 for loading the executable file into system memory for execution, such a direct approach also eliminates a call to _DLL_InitTerm in the loaded DLL that could execute arbitrary and potentially dangerous code, as well as it eliminates searching for and loading all dependend DLLs. This is both faster and much more secure.

The exeinfo API is defined in the libcx/exeinfo.h header and currently allows to query the executable file format and read OS/2 resource objects embedded in such a file. More functionality is planned for future versions.

Notes on `spawn2` API usage.

The spawn2 API provides the following enhancements over the regular spawnvpe function:

Specifying an initial directory for the started executable.
Passing a set of file descriptors to be used as standard I/O of the started executable.
Disabling inheritance of all file descriptors of the parent process with a single flag.
Performing all the above in a completely thread-safe manner.
Supporting P_SESSION, P_PM and their additional flags according to EMX specs.
Returning an actual PID of a process started in P_UNRELATED mode when used together with P_2_THREADSAFE (unique feature, not available in EMX or in DosStartSession API).
Extended file handle redirection via P_2_XREDIR mode which allows for selective file descriptor inheritance including inheriting parent process' file descriptors under different file descriptor numbers in the child process.

The spawn2 API is defined in the libcx/spawn2.h header. Consult it for more details.

Notes on C heap displacement.

Starting with version 0.6.1, LIBCx places the regular C heap (used by default when malloc or calloc is called) in the high memory area above 512 MB. This memory area is available in modern OS/2 versions (4.5 and above) provided that VIRTUALADDRESSLIMIT is set to a value greater than 512 in CONFIG.SYS. Previously, the kLIBC voting process was involved to decide if high memory should be used for the regular C heap: this would happen only if the EXE and all kLIBC DLLs statically linked to it were built with the -Zhigh-mem LDFLAGS option. However, this voting proved to be misleading in many cases (see #48 for details) so a mechanism to override this voting process was built into LIBCx. By default, this mechanism will force the high memory area to be used for the regular C heap if high memory is available (and will fall back to using the low memory area otherwise) regardless of whether there are any DLLs built without -Zhigh-mem. It can also be controlled by the LIBCX_HIGHMEM environment variable as follows:

LIBCX_HIGHMEM=0 forces the low memory area to be used for the regular C heap (for testing purposes).
LIBCX_HIGHMEM=1 forces the high memory area to be used for the regular C heap (the default mode, the same as when this variable is absent).
LIBCX_HIGHMEM=2 restores the original kLIBC voting process and disables any LIBCx intervention (for compatibility with old applications).
LIBCX_HIGHMEM=3 is the same as LIBCX_HIGHMEM=1 but a check is made using the kLIBC voting process if there is any DLL that votes against high memory usage (i.e. it's built without -Zhigh-mem). If there is such a DLL, or if the EXE itself is built without -Zhigh-mem, then LIBC will terminate the process using abort() before entering main() with a respective message written to the standard output. This mode is intended for safety checking purposes.

Note that regardless of where the regular C heap is placed, the application may access any of the specific heaps at any time with the respective kLIBC API calls from <emx/umalloc.h> (i.e. _lmalloc will always allocate from the low memory heap while _hmalloc will allocate from the high memory heap when high memory is available). This is the recommended way to allocate memory if the application knows exactly which kind of memory it needs (low memory is in particular needed for calling some native OS/2 APIs that don't support high memory and don't have SafeDos* wrappers in <os2safe.h>, and also to communicate to OS/2 device drivers). It is also possible to override the type of the regular heap for the current thread by setting a desired heap with the _udefault function -- this will work regardless of the mode forced by LIBCx.

If your application wants to use the low memory heap by default for all new threads regardless of the LIBCX_HIGHMEM setting, then in addition to the _udefault call on the main thread, the low memory heap (which is returned by _linitheap()) should also be assigned to the undocumented _um_regular_heap variable before any other thread is created. The resulting behavior is equivalent to building the application without -Zhigh-mem and not linking it against LIBCx so it makes sense when you need LIBCx functionality but without its heap displacement intervention.

Build instructions

Using LIBCx in your applications

The easiest and the only officially supported way to use LIBCx in your application is to use a binary build provided by bitwise. In the RPM/YUM environment (see the next section) this may be easily achieved by running yum install libcx-devel from the command line and then adding -lcx to your linker options. If you take a ZIP version from bitwise ZIP archives, you will have to resolve all possible dependencies yourself, set proper include paths and link to libcx.a by full name.

Building LIBCx

The development of LIBCx, as well as all other projects maintained by bitwise, relies on the RPM/YUM environment for OS/2 (also maintained by us). This environment, in particular, builds up a UNIXROOT environment on the OS/2 machine and provides recent versions of the Unix tool chain (including the GCC 4.x.x compiler) ported to OS/2, as well as a great number of open-source libraries and a set of OS/2-specific tools needed for the development process. So, in order to build LIBCx, you will need to do the following:

Install RPM/YUM on your existing OS/2 system or upgrade it to ArcaOS which comes with RPM/YUM pre-installed.
Install the basic tool chain with yum install gcc gcc-wlink gcc-wrc libc-devel kbuild kbuild-make rpm-build.
Inspect libcx.spec for specific build requirements and build steps which you can either complete automatically with rpmbuild libcx.spec -bb or manually from the command line.

bitwiseworks/libcx