SJTU-IPADS/librdpma

register mr failed at addr: (0x7f73b1200000,2147483648) with error: Operation not supported

UntaggedRui opened this issue · 1 comments

I want to reproduce the one-sided experiment. When I run ./nvm_server --host 192.168.2.206 --nvm_file=/mnt/pmem0/rui/testfile, the error happend:[handler.hh:67] register mr failed at addr: (0x7f73b1200000,2147483648) with error: Operation not supported. I found this error happend when registering MR, it is located at nvm/benchs/one_sided/server.cc:142,RDMA_ASSERT(ctrl.registered_mrs.create_then_reg( i, rmem, ctrl.opened_nics.query(i).value())) << " reg MR for nic: " << i << " failed"; , When I changed rmem to Arc<RMem>(new RMem(1024*1024*1024*2L)), The program runs successfully.
My nic is Mellanox Technologies MT28800 Family [ConnectX-5 Ex], my hugepage info and nvm info are

 cat /proc/meminfo |grep -i huge
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:    4500
HugePages_Free:     4500
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
 df -h
Filesystem                         Size  Used Avail Use% Mounted on
/dev/pmem0                         244G  103G  129G  45% /mnt/pmem0
/dev/pmem1                         244G  101G  131G  44% /mnt/pmem1

Is this because we need nic support to register the mmaped memory of the NVM area as MR?
what's more, I found dram_region1 at line 96 was not used later so I deleted it because I didn't have 4G of memory for hugepages.

solved.
When I run with sudo ./nvm_server --host 192.168.2.206 --nvm_file=/dev/pmem0, the program runs successfully.