JoranHonig/tarantula

Suspiciousness is either 0 or 1

Closed this issue · 4 comments

@cgewecke adding an issue here so we can figure out wha't going wrong. Continueing from sc-forks/solidity-coverage#430 (comment)

In short:
@cgewecke ran fault localisation on the coverage output for sushiswap, and got the following output:
https://gist.github.com/cgewecke/2859f6e35aeb780f107757a45fd174c8https://gist.github.com/cgewecke/2859f6e35aeb780f107757a45fd174c8

It's weird because there is no gradient in the score, only min or max suspiciousness. Maybe there is a bug in the library that causes this problem.

@cgewecke can you share the mocha & coverage json files that you used as an input?

Yes, gists here:

And my tarantula script was:

const fl = require('tarantula-fl');
const mochaOutput = require('./mochaOutput.json');
const testMatrix = require('./testMatrix.json');

const testData = {
  testResults: fl.fromMocha(mochaOutput),
  coverage: fl.fromSolCover(testMatrix)
}

const score = fl.tarantulaScore(testData)
console.log(JSON.stringify(score, null, ' '));

Also, I published that branch as a beta today. It's installable as

npm install --save-dev solidity-coverage@beta

The mocha JSON is automatically generated with the --matrix flag and written to mochaOutput.json

@cgewecke I found the problem 👌

I had no special treatment for situations where there were no failing tests. Should be fixed with v0.0.7!