Eval Results - Have Distance -1 and Normalized Distance of 0
Closed this issue · 1 comments
https://docs.google.com/spreadsheets/d/1tQ-hq1nEMXG7RC1foqdatau8Fg-zt1yXdgMKFAn0c6I/edit#gid=0
A normalized distance of 0 is misleading because it indicates it was a match.
I think the -1 indicates there was no command.
We should probably be setting normalized distance to 1 in this case.
The bug is here
foyle/app/pkg/eval/evaluator.go
Line 235 in a8b0edd
If there are no code blocks in the response then we don't compute distance.
I think in this case distance is set to its uninitialized value of -1. I think for normalized distance we are defaulting to either the golang or proto default value.
The correct thing to do would be to set distance to be the maximum distance and thus normalized distance is 1.