ChronoDK/GodotBigNumberClass

isLessThan error when exponents differ by 1

Opened this issue · 2 comments

Thanks for creating this class. It's very useful, but I've ran into one error: Currently isLessThan can return the wrong answer if the exponents differ by one.

Steps to reproduce:

var m9e2 = Big.new(9, 2)
var m01e3 = Big.new(0.1, 3)
print(m01e3.isGreaterThan(m9e2))
print(m9e2.isGreaterThan(m01e3))
print(m9e2.isLessThan(m01e3))
print(m01e3.isLessThan(m9e2))

returns:

true
false
true
false

isLessThan simply returns true if exponent < n.exponent and always returns false if exponent > n.exponent.

Possible fix:

## Equivalent of [code]Big < n[/code]
func isLessThan(n) -> bool:
	n = Big._typeCheck(n)
	Big.normalize(n)
	if (mantissa == 0
			and (n.mantissa > MANTISSA_PRECISION or mantissa < MANTISSA_PRECISION)
			and n.mantissa == 0
	):
		return false
	if exponent < n.exponent:
		if exponent == n.exponent - 1 and mantissa > 10*n.mantissa:
			return false
		return true
	elif exponent == n.exponent:
		if mantissa < n.mantissa:
			return true
		return false
	else:
		if exponent == n.exponent + 1 and mantissa * 10 < n.mantissa:
			return true
		return false

I can reliably recreate this issue as well. I have used the code provided to success. @CyranosaurusRekt you should create a pull request with your suggested fix.

It looks like negative numbers don't get accounted for. So if the big number is m -5.123 e 3 and you are checking if it's less than 0 or any positive number less than 5123 it will return false.

I think the solution is that we should just manually verify negative numbers.

func isLessThan(n) -> bool:
	n = Big._typeCheck(n)
	Big.normalize(n)
	if (mantissa == 0
			and (n.mantissa > MANTISSA_PRECISION or mantissa < MANTISSA_PRECISION)
			and n.mantissa == 0
	):
		return false
	if exponent < n.exponent:
		if exponent == n.exponent - 1 and mantissa > 10*n.mantissa:	
			return false #9*10^3 > 0.1*10^4
		return true
	elif exponent == n.exponent:
		if mantissa < n.mantissa:
			return true
		return false
	else:
		if exponent == n.exponent + 1 and mantissa * 10 < n.mantissa:
			return true
		# if it's negative and the other is positive ofcourse it's less regardless of exponent
		if mantissa < 0 and n.mantissa >= 0:
			return true
		return false