Unable to ping/wget to nginx server
dakshinai opened this issue · 6 comments
Hello,
I have setup 1 of 2 NIC cards to use dpdk-2.2 driver. The card has been configured to IP 2.2.2.2. I modified nginx.conf to listen on 2.2.2.2:80. As per the steps, I started opendp and then nginx server. However I'm unable to ping or wget access the nginx server. Debug information on opendp/nginx does not show any abnormalities. Please find inline. Since the card is on dpdk driver I'm unable to trace using wireshark/tcpdump.
Log from opendp
EAL: Detected lcore 71 as core 27 on socket 1
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 72 lcore(s)
EAL: VFIO modules not all loaded, skip VFIO support...
EAL: Setting up physically contiguous memory...
EAL: Ask a virtual area of 0x140000000 bytes
EAL: Virtual area found at 0x7f6f80000000 (size = 0x140000000)
EAL: Ask a virtual area of 0x40000000 bytes
EAL: Virtual area found at 0x7f6f00000000 (size = 0x40000000)
EAL: Ask a virtual area of 0x100000000 bytes
EAL: Virtual area found at 0x7f6dc0000000 (size = 0x100000000)
EAL: Requesting 5 pages of size 1024MB from socket 0
EAL: Requesting 5 pages of size 1024MB from socket 1
EAL: TSC frequency is ~2294681 KHz
EAL: Master lcore 0 is ready (tid=554f68c0;cpuset=[0])
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:01:00.1 on NUMA socket -1
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:81:00.0 on NUMA socket -1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7f70c0000000
EAL: PCI memory mapped at 0x7f70c0080000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:81:00.1 on NUMA socket -1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: Not managed by a supported kernel driver, skipped
param nb 1 ports 1
port id 0
Start to Init port
port 0:
port name rte_ixgbe_pmd:
max_rx_queues 128: max_tx_queues:128
rx_offload_capa 31: tx_offload_capa:63
Creating queues: rx queue number=1 tx queue number=1...
MAC Address:00:1B:21:44:F1:D8
lcore id:0, tx queue id:0, socket id:0
PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f70bfd28dc0 hw_ring=0x7f70bfd2ae00 dma_addr=0x23fd2ae00
PMD: ixgbe_set_tx_function(): Using simple tx code path
PMD: ixgbe_set_tx_function(): Vector tx enabled.
Allocated mbuf pool on socket 0, mbuf number: 16384
Initializing rx queues on lcore 0 ...
port id:0, rx queue id: 0, socket id:0
PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f70bd6561c0 sw_sc_ring=0x7f70bd655c80 hw_ring=0x7f70bd656700 dma_addr=0x23d656700
core mask: 1, sockets number:1, lcore number:1
start to init netdp
USER8: lcore mask 0x1
USER8: lcore id 0 is enable
USER8: lcore number 1
USER8: rnh addr: 0x7f70bd3a0f80, rnh_treetop addr: 0x7f70bd3a1010
USER1: rte_ip_frag_table_create: allocated of 25165952 bytes at socket 0
USER8: UDP layer init successfully, Use memory:4194304 bytes
USER8: TCP hash table init successfully, tcp pcb size 448 total size 58720256
USER8: so shm memory 17039360 bytes, so number 133120, sock shm size 128 bytes
USER8: Sock init successfully, allocated of 42598400 bytes
add eth0 device
add IP 2020202 on device eth0
Show interface
eth0 HWaddr 00:1b:21:44:f1:d8
inet addr:2.2.2.2
inet addr:255.255.255.0
add static route
Destination Gateway Netmask Flags Iface
2.2.2.0 * 255.255.255.0 U C 0
2.2.2.5 * 255.255.255.255 U H L 0
3.3.3.0 2.2.2.5 255.255.255.0 U G 0
USER8: NETDP mgmt thread startup
PMD: ixgbe_set_rx_function(): Port[0] doesn't meet Vector Rx preconditions or RTE_IXGBE_INC_VECTOR is not enabled
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=0.
Checking link status .done
Port 0 Link Up - speed 10000 Mbps - full-duplex
USER8: main loop on lcore 0
USER8: -- lcoreid=0 portid=0 rxqueueid=0
nb ports 1 hz: 2294681988
Log from dpdk-nginx
EAL: Detected lcore 71 as core 27 on socket 1
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 72 lcore(s)
EAL: Setting up physically contiguous memory...
EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the kernel.
EAL: This may cause issues with mapping memory into secondary processes
EAL: Analysing 10 files
EAL: Mapped segment 0 of size 0x140000000
EAL: Mapped segment 1 of size 0x40000000
EAL: Mapped segment 2 of size 0x100000000
EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history> already exists
RING: Cannot reserve memory
EAL: TSC frequency is ~2294681 KHz
EAL: Master lcore 0 is ready (tid=910c18c0;cpuset=[0])
USER8: netdpsock lcore id 0
USER8: netdpsock any lcore id 0xffffffff
2016/02/10 03:08:48 [debug] 44646#0: bind() 0.0.0.0:80 #2005
skip linux fd 5
2016/02/10 03:08:48 [notice] 44646#0: using the "epoll" event method
2016/02/10 03:08:48 [notice] 44646#0: nginx/1.9.5
2016/02/10 03:08:48 [notice] 44646#0: built by gcc 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu2)
2016/02/10 03:08:48 [notice] 44646#0: OS: Linux 4.2.0-19-generic
2016/02/10 03:08:48 [notice] 44646#0: getrlimit(RLIMIT_NOFILE): 1024:65536
2016/02/10 03:08:48 [debug] 44646#0: write: 9, 00007FFF49CC15B0, 6, 0
2016/02/10 03:08:48 [debug] 44646#0: malloc: 0000000000AB3F40:6144
2016/02/10 03:08:48 [debug] 44646#0: malloc: 00007F969108C010:212992
2016/02/10 03:08:48 [debug] 44646#0: malloc: 0000000000AD3CF0:98304
2016/02/10 03:08:48 [debug] 44646#0: malloc: 0000000000AEBD00:98304
2016/02/10 03:08:48 [debug] 44646#0: epoll add event: fd:2005 op:1 ev:00002001
2016/02/10 03:08:48 [debug] 44646#0: worker cycle
2016/02/10 03:08:48 [debug] 44646#0: epoll timer: -1
I also tried ping from another machine on same subnet using ip 2.2.2.3/2.2.2.4 but with no luck.
Please let me know if I need to give further information to help fix this. I had emailed you earlier and you had asked me to use debug info on both opendp and dpdk-nginx.
-Dakshina
You mean that you can't ping 2.2.2.2 successfully from another machine? you can tcpdump the packets to see what packets was droped. maybe some issue with your network connectivity.
maybe your NIC driver parameters isn't correct, you may add some debug log in opendp_main.c file to trace the packets.
I was able to ping across the subnet 2.2.2.0. I was also able to ping dpdk application on server(2.2.2.2). Wireshark stats on server showed tcp SYN requests being re-transmitted until connection times out on nginx server for wget request. Similarly ICMP ping requests on server have no response.
If another IP on subnet 2.2.2.1 is reachable on kernel, I see ICMP errors that 2.2.2.2 is unreachable.
Also there is a connect failure to the nginx server for a wget. Hence I suspect an issue with nginx stack.
2.2.2.2 IP address was already configured in netdp stack, you can't ping successfully from another mechine. maybe your cable connection between netdp stack and another mechine.