Permission denied when exec "sudo ./script/nicctl.sh create_vf 0000:00:1f.6"

Question

Permission denied when exec "sudo ./script/nicctl.sh create_vf 0000:00:1f.6"

Closed this issue a year ago · 15 comments

0000:00:1f.6 'Ethernet Connection (4) I219-V 15d8' if=enp0s31f6 drv=e1000e unused=vfio-pci Active
./script/nicctl.sh: line 63: /sys/bus/pci/devices/0000:00:1f.6/sriov_numvfs: Permission denied

Answer 1 · 2023-08-25T08:29:45.000Z

Hi, seems this NIC doesn't support SRIOV feature. You can use PF PMD directly.

sudo ./script/nicctl.sh bind_pmd 0000:00:1f.6

And then use 0000:00:1f.6 in the json file.

Answer 2 · 2023-08-28T07:54:40.000Z

Thanks for your reply, Then another issue occurred:
./build/app/RxTxApp --config_file config/test_tx_1port_1v.json
st_app_parse_json, using json-c version: 0.16
MT: dev_eal_init(0), port_param: 0000:00:1f.6
MT: dev_eal_init, wait eal_init_thread done
EAL: Detected CPU lcores: 4
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Selected IOVA mode 'VA'
EAL: No free 1048576 kB hugepages reported on node 0
EAL: VFIO support initialized
EAL: Cannot open /dev/vfio/9: Permission denied
EAL: Failed to open VFIO group 9
EAL: Requested device 0000:00:1f.6 cannot be used
EAL: Bus (pci) probe failed.
TELEMETRY: No legacy callbacks, legacy socket not created
MT: st version: 23.12.0 Fri Aug 25 08:29:52 2023 a2adf88 gcc-9.4.0, dpdk version: DPDK 23.03.0
MT: Error: mt_dev_get_socket, failed to locate 0000:00:1f.6. Please run nicctl.sh
MT: Error: mtl_init, get socket fail -19
main, mtl_init fail

Answer 3 · 2023-08-28T08:28:22.000Z

use sudo to run or chang the owner of /dev/vfio/* to current user

Answer 4 · 2023-08-28T08:45:25.000Z

I used sudo to try again, same issue
Is this "2. NIC driver setup" Step necessary, my pci description is "Ethernet Connection (4) I219-V", which driver should I install？ Can the ST2110 lib be used without dpdk ?

Answer 5 · 2023-08-28T12:58:25.000Z

No, MTL is based on DPDK, you must has a NIC supported by DPDK, please refer to https://doc.dpdk.org/guides/nics/ for the supported NIC in DPDK.

You can run DPDK testpmd(https://doc.dpdk.org/guides/testpmd_app_ug/index.html) to verify if you NIC is supported by DPDK or not.

Answer 6 · 2023-08-29T05:24:49.000Z

Thanks for the reply, have you measured the end-to-end delay time with these conditions: 1G network, 1080P, 30fps, codec JPEG XS.

Answer 7 · 2023-08-30T05:09:16.000Z

no, the major latency is the codec part, it depends on the jpegxs codec you used. the network latency is a fixed 33ms(one frame time) here.

Answer 8 · 2023-09-07T06:46:47.000Z

After used sudo ./script/nicctl.sh bind_pmd 0000:00:1f.6
Hi I meet this new issue:
when executing ./build/app/RxTxApp --config_file config/test_tx_1port_1v.json

st_app_parse_json, using json-c version: 0.16
MT: dev_eal_init(0), port_param: 0000:00:1f.6
MT: dev_eal_init, wait eal_init_thread done
EAL: Detected CPU lcores: 4
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Selected IOVA mode 'VA'
EAL: No free 1048576 kB hugepages reported on node 0
EAL: VFIO support initialized
EAL: Cannot open /dev/vfio/9: Permission denied
EAL: Failed to open VFIO group 9
EAL: Requested device 0000:00:1f.6 cannot be used
EAL: Bus (pci) probe failed.
TELEMETRY: No legacy callbacks, legacy socket not created
MT: st version: 23.12.0 Thu Sep 7 13:50:29 2023 e413db3-dirty gcc-9.4.0, dpdk version: DPDK 23.07.0
MT: Error: mt_dev_get_socket, failed to locate 0000:00:1f.6. Please run nicctl.sh
MT: Error: mtl_init, get socket fail -19
main, mtl_init fail

Answer 9 · 2023-09-08T01:13:50.000Z

Like I said before, please grant /dev/vfio/* access to current user. Or just use sudo to run the RxTxApp.

Answer 10 · 2023-09-08T01:45:10.000Z

Still some issue,
sudo ./build/app/RxTxApp --config_file config/test_tx_1port_1v.json
st_app_parse_json, using json-c version: 0.16
MT: dev_eal_init(0), port_param: 0000:00:1f.6
MT: dev_eal_init, wait eal_init_thread done
EAL: Detected CPU lcores: 4
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Selected IOVA mode 'VA'
EAL: No free 1048576 kB hugepages reported on node 0
EAL: VFIO support initialized
EAL: 0000:00:1f.6 VFIO group is not viable! Not all devices in IOMMU group bound to VFIO or unbound
EAL: Requested device 0000:00:1f.6 cannot be used
EAL: Bus (pci) probe failed.
TELEMETRY: No legacy callbacks, legacy socket not created
MT: st version: 23.12.0 Thu Sep 7 16:44:19 2023 e413db3 gcc-9.4.0, dpdk version: DPDK 23.07.0
MT: Error: mt_dev_get_socket, failed to locate 0000:00:1f.6. Please run nicctl.sh
MT: Error: mtl_init, get socket fail -19
main, mtl_init fail

Answer 11 · 2023-09-08T03:11:32.000Z

EAL: 0000:00:1f.6 VFIO group is not viable! Not all devices in IOMMU group bound to VFIO or unbound

Please check your VFIO partition and make sure all devices under the IOMMU group are bound to VFIO.

Another option is disable IOMMU and run RxTxApp with root permission.

Answer 12 · 2023-09-26T10:46:30.000Z

After disabed IOMMU
sudo ./script/nicctl.sh bind_pmd 0000:00:1f.6
0000:00:1f.6 'Ethernet Connection (4) I219-V 15d8' unused=e1000e,vfio-pci
Warn: no iommu_groups in /sys/kernel/iommu_groups/
Warn: IOMMU is not enabled on this setup, please check kernel command line and BIOS settings
Error: bind failed for 0000:00:1f.6 - Cannot bind to driver vfio-pci: [Errno 22] Invalid argument
Bind bdf: 0000:00:1f.6 to vfio-pci succ

Answer 13 · 2023-09-26T10:57:12.000Z

sudo bash -c 'echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode'

Answer 14 · 2023-09-26T10:59:21.000Z

EAL: Error disabling MSI-X interrupts for fd 24

Answer 15 · 2023-10-08T08:08:57.000Z

If you still has problem to run with DPDK pmd, please try to run with the fallback path(https://github.com/OpenVisualCloud/Media-Transport-Library/blob/main/doc/kernel_socket.md) recently introduced.