[instancer] Rounding differences between harfbuzz and fonttools
Closed this issue · 6 comments
While updating some test expectations I ran into several cases of rounding differences between harfbuzz and fonttools (#4641 ). Let's see if we can track them down and fix.
Example reproduction:
fonttools varLib.instancer Roboto-Variable.ABC.ttf wght=200:300:500 wdth=80:90 --no-recalc-timestamp -o instanced.ttf
fonttools subset instanced.ttf --unicodes="*" --recalc-bounds --no-prune-codepage-ranges --recalc-average-width --output-file=ft.ttf
util/hb-subset Roboto-Variable.ABC.ttf --instance="wght=200:300:500,wdth=80:90" --optimize --unicodes="*" -o hb.ttf
ttx hb.ttf ft.ttf
diff -u ft.ttx hb.ttx
Using this file:
Roboto-Variable.ABC.zip
I was able to eliminate the diffs in the above example by switching the solver and some of the tuple var store code over to use double's instead of floats:
If perf is not affected I'm fine with this I think.
Sounds good, I'll get some perf numbers and see where it lands.
Please replace the "#define float double" with actual doubles, and add a link to this issue.
Definitely, I'll make a proper change if the perf looks good.
Should be fixed for now by: #4665
I'll close for now but reopen if I encounter any more rounding differences in the future.