seattlerb/flay

No secondary sort of files with same score

njhammond opened this issue · 5 comments

If files have the same score then default sort algorithm on different machine architectures may output different results.

Found when comparing results of flay from Linux machine to M1 Macbook of same project

Example output, machine 1:

33:00: c.rb
33:00: b.rb
33:00 a.rb

Machine 2:

33.00: b.rb
33.00: a.rb
33:00: c.rb

Fix:

Line 487 of ./lib/flay.rb is
self.summary.sort_by { |_,v| -v }.each do |file, score|

Replace with

  self.summary.sort_by { |f,v| [-v, f] }.each do |file, score|

This will make sure flay is sorted by score then alphabetical by file name.

Good find! Not sure why you're comparing results across platforms, but good find nonetheless!

It's mostly dependent upon the filesystem ordering of globs... so it could probably also be addressed by sorting all input files before processing.

I suppose they are sorted today by default.
https://bugs.ruby-lang.org/issues/8709

I suppose they are sorted today by default.

https://bugs.ruby-lang.org/issues/8709

I would assume so... But I'm also sorting this by the float score so being explicit about ties can't hurt (much).

Good find! Not sure why you're comparing results across platforms, but good find nonetheless!

It's mostly dependent upon the filesystem ordering of globs... so it could probably also be addressed by sorting all input files before processing.

Reasons why we are comparing results:

Ruby project. Multiple developers.

One developer is running Linux. Generates flay output for a project, checks it in to Github.

Another developer is running M1 Mac. Runs flay on same source code. Gets different output so updates Github entry.

Our log files were showing continuous update of flay reports each time a different person ran flay.

I wouldn't recommend checking in generated output (of pretty much any sort). but this should be handled now.