register mr failed at addr: (0x7f73b1200000,2147483648) with error: Operation not supported
UntaggedRui opened this issue · 1 comments
I want to reproduce the one-sided experiment. When I run ./nvm_server --host 192.168.2.206 --nvm_file=/mnt/pmem0/rui/testfile
, the error happend:[handler.hh:67] register mr failed at addr: (0x7f73b1200000,2147483648) with error: Operation not supported
. I found this error happend when registering MR, it is located at nvm/benchs/one_sided/server.cc:142
,RDMA_ASSERT(ctrl.registered_mrs.create_then_reg( i, rmem, ctrl.opened_nics.query(i).value())) << " reg MR for nic: " << i << " failed";
, When I changed rmem
to Arc<RMem>(new RMem(1024*1024*1024*2L))
, The program runs successfully.
My nic is Mellanox Technologies MT28800 Family [ConnectX-5 Ex]
, my hugepage info and nvm info are
cat /proc/meminfo |grep -i huge
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
HugePages_Total: 4500
HugePages_Free: 4500
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/pmem0 244G 103G 129G 45% /mnt/pmem0
/dev/pmem1 244G 101G 131G 44% /mnt/pmem1
Is this because we need nic support to register the mmaped memory of the NVM area as MR?
what's more, I found dram_region1
at line 96 was not used later so I deleted it because I didn't have 4G of memory for hugepages.
solved.
When I run with sudo ./nvm_server --host 192.168.2.206 --nvm_file=/dev/pmem0
, the program runs successfully.