/xpmem

Linux Cross-Memory Attach

Primary LanguageCGNU Lesser General Public License v2.1LGPL-2.1

***************************************************************************
***************************************************************************
***************************************************************************

If you wish to contribute to or want to see the latest version of XPMEM
the new repository is @ http://gitlab.com/hjelmn/xpmem. I do not intent
to continue to use github for this repository.

***************************************************************************
***************************************************************************
***************************************************************************



This is an experimental version of XPMEM based on a version provided by
Cray and uploaded to https://code.google.com/p/xpmem. This version supports
any kernel 3.12 and newer. Keep in mind there may be bugs and this version
may cause kernel panics, code crashes, eat your cat, etc.

XPMEM is a Linux kernel module that enables a process to map the
memory of another process into its virtual address space. Source code
can be obtained by cloning the Git repository, original Mercurial
repository or by downloading a tarball from the link above.

The XPMEM API has three main functions:

  xpmem_make()    
  xpmem_get()
  xpmem_attach()

A process calls xpmem_make() to export a region of its virtual address
space. Other processes can then attach to the region by calling
xpmem_get() and xpmem_attach(). After a memory region is attached, it
is accessed via direct loads and stores. This enables upper-level
protocols such as MPI and SHMEM to perform single-copy address-space
to address-space transfers, completely at user-level.

Note, there is a limitation to the usage of an attached region. Any
system call that will call get_user_pages() on the region from the
non-owning process with get EFAULT. This include pthread mutexes
and condition variable, and SYS V semaphores. We intend to address
this limitation in a future release.

XPMEM regions are free to have "holes" in them, meaning virtual memory
regions that are not allocated. This makes XPMEM somewhat more
flexible than mmap(). A process could, for example, export a region
via XPMEM starting at address 0 and extending 4 GB. Accesses to
allocated (valid) virtual addresses in this region proceed normally,
and pages are mapped between address spaces on demand. A segfault will
occur if the source process or any other process mapping the region
tries to access an unallocated (invalid) virtual address in the
region.