isLessThan error when exponents differ by 1
Opened this issue · 2 comments
Thanks for creating this class. It's very useful, but I've ran into one error: Currently isLessThan can return the wrong answer if the exponents differ by one.
Steps to reproduce:
var m9e2 = Big.new(9, 2)
var m01e3 = Big.new(0.1, 3)
print(m01e3.isGreaterThan(m9e2))
print(m9e2.isGreaterThan(m01e3))
print(m9e2.isLessThan(m01e3))
print(m01e3.isLessThan(m9e2))
returns:
true
false
true
false
isLessThan simply returns true if exponent < n.exponent
and always returns false if exponent > n.exponent
.
Possible fix:
## Equivalent of [code]Big < n[/code]
func isLessThan(n) -> bool:
n = Big._typeCheck(n)
Big.normalize(n)
if (mantissa == 0
and (n.mantissa > MANTISSA_PRECISION or mantissa < MANTISSA_PRECISION)
and n.mantissa == 0
):
return false
if exponent < n.exponent:
if exponent == n.exponent - 1 and mantissa > 10*n.mantissa:
return false
return true
elif exponent == n.exponent:
if mantissa < n.mantissa:
return true
return false
else:
if exponent == n.exponent + 1 and mantissa * 10 < n.mantissa:
return true
return false
I can reliably recreate this issue as well. I have used the code provided to success. @CyranosaurusRekt you should create a pull request with your suggested fix.
It looks like negative numbers don't get accounted for. So if the big number is m -5.123 e 3 and you are checking if it's less than 0 or any positive number less than 5123 it will return false.
I think the solution is that we should just manually verify negative numbers.
func isLessThan(n) -> bool:
n = Big._typeCheck(n)
Big.normalize(n)
if (mantissa == 0
and (n.mantissa > MANTISSA_PRECISION or mantissa < MANTISSA_PRECISION)
and n.mantissa == 0
):
return false
if exponent < n.exponent:
if exponent == n.exponent - 1 and mantissa > 10*n.mantissa:
return false #9*10^3 > 0.1*10^4
return true
elif exponent == n.exponent:
if mantissa < n.mantissa:
return true
return false
else:
if exponent == n.exponent + 1 and mantissa * 10 < n.mantissa:
return true
# if it's negative and the other is positive ofcourse it's less regardless of exponent
if mantissa < 0 and n.mantissa >= 0:
return true
return false