README ========== 0. Prerequisites 1. Preparing build 2. Configuring NGINX 3. Running NGINX 4. Limitations or Known Issues 5. Changelog 0. Prerequisites ================ 0.1. Get latest OFP to your $PATH_OFP_ROOT and run ./scripts/devbuild_ofp_odp_dpdk.sh This builds and installs DPDK, ODP-DPDK and OFP. 0.2. Configure DPDK: ls /sys/devices/system/node/ sudo sh -c 'echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages' sudo sh -c 'echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages' sudo mkdir /mnt/huge sudo mount -t hugetlbfs nodev /mnt/huge sudo /sbin/modprobe uio ulimit -Sn 2048 insmod scripts/dpdk/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko Configure interfaces: ./tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver ============================================ <none> Network devices using kernel driver =================================== 0000:02:00.0 'I350 Gigabit Network Connection' if=p1p1 drv=igb unused=igb_uio *Active* 0000:02:00.1 'I350 Gigabit Network Connection' if=p1p2 drv=igb unused=igb_uio 0000:81:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=eth0 drv=ixgbe unused=igb_uio *Active* 0000:81:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=eth1 drv=ixgbe unused=igb_uio *Active* Other network devices ===================== <none> $ sudo ifconfig eth0 down sudo ./tools/dpdk_nic_bind.py -u 0000:01:00.0 sudo ./tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0 $ ./tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver ============================================ 0000:81:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused= 0000:81:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused= Network devices using kernel driver =================================== 0000:02:00.0 'I350 Gigabit Network Connection' if=p1p1 drv=igb unused=igb_uio *Active* 0000:02:00.1 'I350 Gigabit Network Connection' if=p1p2 drv=igb unused=igb_uio Other network devices ===================== <none> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PATH_OFP_ROOT/scripts/check-odp/new-build/lib/ export ODP_PLATFORM_PARAMS="-n 4 -- -p 4" 1. Preparing build ================== set build variables with respective OFP and ODP build path (see 0.1.) export OFP_PATH=$PATH_OFP_ROOT/scripts/check-odp/new-build/ export OFP_ODP=$PATH_OFP_ROOT/scripts/check-odp/new-build/ export RTE_SDK=$PATH_OFP_ROOT/scripts/dpdk/ export RTE_TARGET=x86_64-native-linuxapp-gcc ./configure --sbin-path=/usr/local/nginx_dpdk/nginx --conf-path=/usr/local/nginx_dpdk/nginx.conf --pid-path=/usr/local/nginx_dpdk/nginx.pid --with-http_ssl_module --with-select_module [--without-http_rewrite_module] make install Note: make clean will remove main Makefile and obj dir. So configure shall be run again. It would be useful to save the above build steps in a script and run the script. 2. Configuring NGINX ==================== After build and install, NGINX configuration file will be at /usr/local/nginx_dpdk/nginx.conf. To get NGINX debug logs uncomment the debug line error_log logs/error.log debug To run more on more cores increase the number of worker processes: worker_processes number; For now, one also needs to change in code: --- a/src/event/modules/ngx_ofp_module.c +++ b/src/event/modules/ngx_ofp_module.c @@ -44,7 +44,7 @@ #include <sys/syscall.h> #include <dlfcn.h> -#define NUM_QUEUES 1 +#define NUM_QUEUES 3 #define ODP_FD_BITS 30 #include <getopt.h> Debug logs will be in /usr/local/nginx_dpdk/logs/error.log 3. Running NGINX ================ Be sure you have configured DPDK and interfaces as explained at 0.2. In conf/ofp.conf file, configure the ip address for fp0 interface. To start webserver: sh start_nginx.sh 0 To stop webserver: sh stop_nginx.sh 0 From some other machine try to ping the IP we have set for fp0 interface. If ping works, then open a web browser and try to access the same ip address. If web page loads successfully, webserver with OFP is working. 4. Limitations or Known Issues ============================== a) Current NGINX/OFP implementation assumes interface as 0. It will not work for other interfaces. b) Running with keepalive timeout other than zero causing issue while more connection requests are triggered. To avoid this Configure keepalive_timeout to zero in conf file. 5. Changelog ============ Made changes to utilise the OFP event signaling support for handling socket operation like accept and recieve calls on a fd. In ngx_select_module.c file, macro OFP_NOTIFY is enabled to use this implementation. If macro is set to zero, can use older select() implementation.