InseeFr/Trevas

Calculation accuracies

Opened this issue · 2 comments

@noahboerger reported:

  • In trevas the result of an operation is kept in double accuracy (e.g. numeric/power_1)
  • User tests assume sometimes less accurate results (e.g. user test: 8.006001 trevas: 8.006000999999998)

I think every language will give this kind of result. To be sure of the result, if a particular precision is needed, user can use round operator. @hadrienk, can you confirm?

I fully agree on this. This is not an issue on trevas, but more on the testcases we have been using. I think these should be adjusted to use the rounding operation where necessary.

@NicoLaval yes, rounding is the solution. This is a case of the good old 0.1 + 0.2 != 0.3.
https://0.30000000000000004.com/