sqrt() is not fully compliant

[samhocevar]

RIght now our implementation of sqrt() uses the C++ math library’s implementation, but that creates a few differences with PICO-8. Here is a list:

• lots of off-by-one bit differences
• in PICO-8 anything larger than 32761 will return 181
• also PICO-8 returns stuff for negative numbers

[samhocevar]

Some fixed-point code that’s closer to the real thing (still not accurate):

    static inline fix32 sqrt(fix32 x)
    {
        fix32 one(1);
        bool small = x < one;

        if (!x)
            return x;
        if (x >= fix32(181*181))
            return fix32(181);
        if (small)
            x = one / x;

        fix32 y = 0;
        for (fix32 bits = 128; bits >= one; bits /= 2)
            if ((y + bits) * (y + bits) <= x)
                y += bits;

        while (fix32 error = (x - y * y) / (y + y))
            y += error;

        return small ? one / y : y;
    }

[samhocevar]

Good news everyone:

• PICO-8 now has a better sqrt which doesn’t act weird with negative values or values larger than 181².
• z8lua now uses the same algorithm.