This repo holds information on our SIGMETRICS 2023 submission.
TODOs are tracked in Todoist.
We will aim to be both Repeatable and Reproducible. Whether or not the results are Replicatable will be up to future researchers.
A researcher can reliably repeat her own computation.
An independent group can obtain the same results using the author's artifacts.
The following steps should be easily accomplished by the artifact evaluation committee:
- Acquire datasets
- Run experiments*
- Validate experimental results
- Plot data
* Note on running experiments: The experiments will likely take a long time to run. We will need to decide which of the following steps to take (possibly a combination of them):
- Provide cluster time
- Provide a reduced set of experiments to run (allow for larger error bars, run subset of apps, etc.)
- Provide expected experimental output data
We must provide high quality documentation for all tools associated with the project.
We need to provide a DOI or citation for all datasets.