Quenty/NevermoreEngine

round isn't correct

Dollar-500 opened this issue · 2 comments

in qMath, shouldn't Round be like this instead of what it is now? I was pretty sure that I was right but I did some tests just in case cus you're way more experienced than me 😉 and I found that your method doesn't work properly... here's what it should be:

function lib.Round(number, base)
	base = base or 1
	return (math.floor((number*base)+0.5)/base)
end

Thank you for your feedback! I've looked into the issue. However, I think it's working as intended.

I believe this is working as intended. For example, `round(0.923, 0.01) should round to 0.92. The original code does this, while the new code rounds to 0.

local function round_orig(number, base)
	base = base or 1
	return (math.floor((number/base)+0.5)*base)
end

function round_new(number, base)
	base = base or 1
	return (math.floor((number*base)+0.5)/base)
end

print(round_orig(0.923, 0.01)) --> 0.92
print(round_new(0.923, 0.01)) --> 0

Potentially base is a bad name. I'm going to refactor this to precision.

Oh wow, how did I not notice this! In my round function, "precision" is actually 10 to the power of something. for example:

function round(num, precision)
	precision = 10^(precision or 0)
	return math.floor(num * precision + 0.5) / precision
end

print(round(0.923, 2)) --> 0.92

It's just the inverse of what you're doing.