InsightSoftwareConsortium/ITKPerformanceBenchmarking

Historical performance testing refactoring

Closed this issue · 3 comments

dzenanz commented

Outstanding issues

  • change to the data reports for easier parsing
  • easier running of historical performance
  • mechanism to automatically upload data reports to a common repo for everyone to analyze.
jhlegarreta commented

I'd dare to say that the command line that @hjmjohnson shared to run the benchmark:

cd /home/johnsonhj/Dashboard/src/ITK && \
for hash_to_test in $( git log v4.12.0..origin/master --reverse --merges --format="%H" ); do 
/home/johnsonhj/Dashboard/src/ITKPerformanceBenchmarking/ITK_PerformanceTestingDriver.sh \
--src /home/johnsonhj/Dashboard/src/ITK \
--bld /home/johnsonhj/Dashboard/src/ITK-gcc5 \
--perf-src /home/johnsonhj/Dashboard/src/ITKPerformanceBenchmarking \
--perf-bld /home/johnsonhj/Dashboard/src/ITKPB-bld \
--git-tag $
{hash_to_test}
;
done

assumes that the user has its origin up-to-date, so when this piece of code is put into a script, I'd add

git fetch upstream --tags

at the beginning and then walk the tags from v4.12.0..latest, if the lastest can be somehow identified.

thewtex commented

We can now easily run historical performance builds with python ./evaluate-itk-performance.py run --rev-list [...] and upload results to a common data.kitware.com folder with python ./evaluate-itk-performance.py upload [...].

jhlegarreta commented

Wow ! 💯

Awesome Matt. Thanks for doing this. We are getting there.

  • ❤️1