ofiwg/librdmacm

Does librdmacm support register memory like ibv_reg_mr(pd, addr, len, 0)?

Keepmoving-ZXY opened this issue · 3 comments

In my case, the memory comes from mmap and during mmap, the prot argument is PROT_READ and rdma_reg_msgs always report an error. I think the reason is that rdma_reg_msgs requires the memory can be write and read, but the mmap don't give the write permission.
So is there any possible that mmap only gives read permission and some register API in librdmacm support register the readonly memory?

And below is my code, the rdma_create_ep is hard to use, it always return invalid argument error, could you help me debug this?

  int num_device = 0;
  struct ibv_device *device = NULL;
  struct ibv_device **device_list = NULL;
  device_list = ibv_get_device_list(&num_device);
  if (num_device == 0) {
    LOG_ERROR("%s", "no rdma device found");
    retval = -1;
    goto out;
  }

  device = *device_list;
  struct ibv_context *context = NULL;
  context = ibv_open_device(device);
  if (context == NULL) {
    LOG_ERROR("%s", "ibv_open_device failed.");
    retval = -1;
    goto out;
  }

  struct ibv_pd *pd = ibv_alloc_pd(context);
  if (pd == NULL) {
    LOG_ERROR("ibv_alloc_pd failed, %s", strerror(errno));
    retval = -1;
    goto out;
  }

  struct ibv_cq *cq = NULL;
  cq = ibv_create_cq(context, 10, NULL, NULL, 0);
  if (cq == NULL) {
    LOG_ERROR("ibv_create_cq failed, %s", strerror(errno));
    retval = -1;
    goto out;
  }

  memset(&attr, 0, sizeof attr);
  attr.send_cq = cq;
  attr.recv_cq = cq;
  attr.cap.max_send_wr = 10;
  attr.cap.max_recv_wr = 10;
  attr.cap.max_send_sge = 1;
  attr.cap.max_recv_sge = 1;

  ret = rdma_create_ep(&id, res, pd, &attr);
  if (ret) {
    LOG_ERROR("rdma_create_ep failed, %s", strerror(errno));
    retval = -1;
    goto out;
  }

  struct rdma_conn_param cm_params;
  memset(&cm_params, 0, sizeof(cm_params));
  ret = rdma_connect(id, &cm_params);
  if (ret) {
    LOG_ERROR("rdma_connect failed, %s", strerror(errno));
    retval = -1;
    goto out;
  }

Your questions would be better directed to the linux-rdma mailing list. This is not the main project for librdmacm.

You can use ibv_reg_mr(), and other libibverbs calls, with librdmacm calls.

ok, thank you.