JuliaMath/Decimals.jl

Division of decimals gives incorrect result

pnalysz-ipl opened this issue · 2 comments

script:

using Decimals

f1 = 274.94
f2 = 275.67
println(f1/f2)

d1 = Decimals.Decimal(f1)
d2 = Decimals.Decimal(f2)
println(d1/d2)

output:

0.9973519062647368
0.00997351906264736805
hurak commented

Obviously, the culprit is the computation of the inversion - the inv function from the current Decimals package:

julia> y = 275.67
275.67

julia> x = Decimal(y)
Decimal(0, 27567, -2)

julia> 1/y
0.003627525664744078

julia> inv(x)
Decimal(0, 3627525664744078, -20)

julia> print(inv(x))
0.00003627525664744078

In particular, the computation of the resulting quotient q on line 31 in https://github.com/JuliaMath/Decimals.jl/blob/master/src/arithmetic.jl is not correct. It appears to me that a proper code should be

function Base.inv(x::Decimal)
    #str = string(x) # not needed
    #if str[1] == '-'
    #    str = str[2:end]
    #end
    # b = ('.' in str) ? length(split(str, '.')[1]) : 0 # not needed at all
    c = round(BigInt(10)^(-x.q + DIGITS) / x.c)
    #q = (x.q < 0) ? 1 - b - DIGITS : -b - DIGITS # incorrect, b not needed, not sure about 1
    q = -DIGITS
    normalize(Decimal(x.s, c, q))
end

I will play a bit more with it and can submit PR.

hurak commented

I also propose adding another test for inversion that catches the problem discussed in this Issue:

@test inv(Decimal(0, 123, -1)) == Decimal(0, 813008130081300813, -19) # 1/12.3 ≈ 0.08