This repository contains the artifacts for our SIGCOMM '24 paper titled "Bad Packets Come Back, Worse Ones Don't."
Table of Contents:
The artifacts software is developed using C++ and Python3. This software requires Ubuntu 22.04.4 LTS, as it has been tested on this platform. Before proceeding with the installation, please ensure the following dependencies are installed on your system:
Open a terminal and execute the following commands to install cmake, g++, python3, and gnuplot:
sudo apt update
sudo apt install cmake g++ python3 gnuplotTo install the required Python dependencies, we recommend setting up a virtual environment. After that, run:
pip3 install -r requirements.txtIf you encounter any issues with the installation or find that a required dependency is missing, please contact at p.gkigkis (at) cs.ucl.ac.uk for assistance.
Before installing the ns-3 simulator, please ensure that you have installed the necessary dependencies as outlined in the previous steps. It is essential to use the provided version of the ns-3 simulator (version 3.40) because we have made modifications to the simulator core to enable packet drops and random link loss.
To install the simulator navigate to ns3-simulations:
cd ns3-simulationsRun the configuration script to set up the simulator environment:
./ns3 configureFinally, build the ns-3 simulator by executing the following command:
./ns3 buildAfter following these steps, the ns-3 simulator should be installed and ready to use. If you encounter any issues during installation, please contact us.
The artifacts are divided into two main sections: theoretical analysis and NS-3 simulations.
The Theoretical Analysis section provides the necessary tools to generate the results discussed in Section 5.1, particularly generating Figure 7.
The NS-3 Simulations section provides the necessary tools to reproduce the results presented in Section 6 (Evaluation), specifically for generating Figures 8 and 9.
For the theoretical analysis, we have developed a single bash script that executes the simulator and generates Figure 7 from our paper. Please note that the simulator's execution time can vary significantly, ranging from 30 minutes to a few hours. Additionally, due to the randomness in the simulation process, the generated figure might exhibit minor differences based on the seed and random function implementation.
To ensure exact reproducibility, we have included the files generated by the simulator during our run. By using these files, you can reproduce the exact figure presented in the paper. These files are located in the theoretical/paper-results/tempFiles directory.
To reproduce the theoretical analysis, please follow these steps:
Navigate to the theoretical-analysis folder:
cd theoretical-analysisExecute the simulation script:
bash run.shUpon successful execution of the script, you will find the following two plots in the results folder:
closed-loop-theoretical-5perc.pdf: Generated from the output of the simulator you just executed.closed-loop-theoretical-5perc-paper.pdf: Generated from the provided files from our simulation execution.
For the evaluation, we used NS-3 simulator (version 3.40). As part of the artifacts we provide a lightweight implementation of Penny in C++.
You can find the source code for the lightweight implementation of Penny in the following directory:
ns3-simulations/scratch/penny.
We provide an easy way to configure the Penny, NS-3 topology, and simulation parameters using JSON-formatted files.
The configuration files can be found in folder ns3-simulations/scratch/penny/configs.
In the provided simulations, we use a dumbbell topology with two routers connected by a bottleneck link.
In this section, we provide detailed instructions on how to generate Figure 8 from the paper.
To do this, first navigate to the ns3-simulations folder:
cd ns3-simulationsNotes:
-
For the simulations, we recommend conducting at least 500 runs.
-
Due to the randomness in the simulation process and the number of executions, the generated figures may exhibit minor differences.
Execute the following command to start the simulations for Figure 8. Replace <number_of_parallel_runs> and <number_of_experiments> with the appropriate values for your setup:
bash experiments/figure8a.sh <number_of_parallel_runs> <number_of_experiments> bash experiments/figure8b.sh <number_of_parallel_runs> <number_of_experiments> bash experiments/figure8c.sh <number_of_parallel_runs> <number_of_experiments> To generate the plots for Figure 8 execute the following commands:
python3 pyscripts/plotAccuracy.py -f tempResults/accuracyOnlyClosedLoop -o plots/accuracyOnlyClosedLoop.pngpython3 pyscripts/plotAccuracy.py -f tempResults/accuracyMixedEqual,tempResults/accuracyMixedEqualWithDup -o plots/accuracyMixedClosedNotClosedLoops.pngpython3 pyscripts/plotAccuracy.py -f tempResults/accuracyOnlyNotClosedLoop -o plots/accuracyOnlyNotClosedLoop.pngNote: For Figure 8b, you can also generate figures for the two individual scenarios:
- The not closed-loop flows do not send any duplicates.
python3 pyscripts/plotAccuracy.py -f tempResults/accuracyMixedEqual -o plots/accuracyMixedEqual.png- The not closed-loop flows send duplicates at a rate of 14.9%.
python3 pyscripts/plotAccuracy.py -f tempResults/accuracyMixedEqualWithDup -o plots/accuracyMixedEqualWithDup.pngIn this section, we provide detailed instructions on how to generate Figure 9 from the paper.
To do this, first make sure that you are inside the ns3-simulations folder.
Notes:
-
For the simulations for figure 9a, we recommend conducting at least 400 experiments.
-
For the simulations for figure 9b, we recommend conducting at least 1000 experiments.
-
Running this minimum number of experiments will produce figures that follow the trends presented in the paper. Note that in the paper, we used 10,000 runs.
-
Due to the randomness in the simulation process and the number of executions, the generated figures may exhibit minor differences.
Execute the following command to start the simulations for Figure 9. Replace <number_of_parallel_runs> and <number_of_experiments> with the appropriate values for your setup:
Notes:
- In this experiment, as we simulate multiple TCP flows as background traffic, the minimum requirements per experiment are
500 MB of disk spaceand2 GB of memory. - After the
.shscript finishes, ensure that all./ns3instances have completed, as some may still be running in the background.
bash experiments/figure9a.sh <number_of_parallel_runs> <number_of_experiments>bash experiments/figure9b.sh <number_of_parallel_runs> <number_of_experiments>To generate the plots for Figure 9 execute the following commands:
python3 pyscripts/plotAggrPerf.py -f tempResults/aggregatesFlowPerformance -o plots/aggregatesFlowPerformance.png -p 1000python3 pyscripts/plotIndivPerf.py -f tempResults/indivFlowPerformance -o plots/indivFlowPerformance.png -p 1000For any questions or issues, please contact Petros Gigis at (p.gkigkis (at) cs.ucl.ac.uk).