eteran/cpp-utilities

The remainder calculation seems to be incorrect

Jayatubi opened this issue · 2 comments

Jayatubi commented

cpp-utilities/fixed/include/eteran/cpp-utilities/Fixed.h

Line 140 in 4c7d8fc

remainder = Fixed<I,F>::from_base(next_to_base<base_type>(t % denominator.to_raw()));

Why the remainder is calculated by the t and denominator? After t <<= fractional_bits the value of t would be much larger than the denominator and would cause an incorrect value of remainder.

eteran commented

the result of t % denominator.to_raw() is then passed to Fixed<I,F>::from_base which I believe handles the scaling correctly.

Do you have an example of it producing the wrong answer?

eteran commented

Closing due to lack of follow up. Please re-open if you have an example demonstrating the issue.