use the faster ryu code for converting double to string
johnynek opened this issue · 11 comments
Here is an efficient port of this algorithm to Scala.
Feel free to pick and adopt it in your codebase.
PS: I copied the java from ryu into my codebase that was writing CSVs (several GBs worth) and the whole end-to-end time got 2x faster, so even in a real system doing other real work, the win is very significant.
Do we do a lot of Double => String in this project? I think this is a wonderful idea, but more applicable on the rendering side, which mostly happens in the downstream libraries. We might get the most benefit implementing this in circe.
That said, it could speed up rendering in jawn-ast.
I was thinking of rendering in jawn-ast, I don't know if circe uses that code, but I thought it might. If not, we'd have to do it in both places.
It looks like jar distribution is stuck (ulfjack/ryu#93). It would make this a lot easier for both projects if they could just add a one-liner to the build and a one-liner to the source.
Yes, I realize I'm talking to Scala's foremost advocate of source dependencies. 😄
Have you considered an option to do it in JDK immediately?
I think we should just copy the code. That’s what I did.
PS: yes it would be good if someone could send this as a PR to the JDK but that would still take years to get deployed and also not help scalajs or native.
Scala.js will not change its implementation of toString, and moreover currently it is impossible to make it run on par with JS implementation in browsers and Node.js.
Scala Native already uses Ryu, but it just a copy of the original Java version from the author of algorithm which is slower than current JDK's toString for whole numbers and numbers with a short mantissa.
You are right that implementation in JDK can take years and almost impossible to back-port it into 8 and 11 versions. But using Long.multiplyHigh intrinsics and internal API of String it could be the most efficient implementation.
Please, peek the latest version of jsoniter-scala and using this implementation of stringify: Double => String function compare it with other implementations for different kind of numbers.
More efficient the Schubfach algorithm is coming to OpenJDK 15 and probably will be ported back to JDK 11. It works almost as Ryū but without rounding loops which affects the speed of serialization for numbers with a short decimal mantissa in latter. Results with the currently proposed patch are quite promising: plokhotnyuk/jsoniter-scala#531
JFYK, the Schubfach way to render doubles (and floats) have been adopted by jsoniter-scala and is available since the v2.2.5 release.
Ironically the best speed up (about 2x times comparing with the previous implementation based on the Ryu algorithm) was got in the Scala.JS version of the library.