tompazourek/Rationals

How do your print decimal form with more than 6 decimals?

NullVoxPopuli opened this issue · 3 comments

NullVoxPopuli commented

For Example:

Best Ask: 0.0001717, Best Bid 0.0001715

Best Ask: 1717/10000000, Best Bid 343/2000000

Best Ask: 0.000172, Best Bid 0.000172

generated by:

        let bestAsk = last10Asks |> Array.last
        let bestAskD = Rational.ParseDecimal (bestAsk, tolerance = 0.000000000001m)

        let bestBid = first10Bids.[0]
        let bestBidD = Rational.ParseDecimal (bestBid, tolerance = 0.000000000001m)

        printfn "\nBest Ask: %s, Best Bid %s" (bestAsk) (bestBid)
        printfn "\nBest Ask: %A, Best Bid %A" (bestAskD) (bestBidD)
        printfn "\nBest Ask: %f, Best Bid %f" ((decimal) bestAskD) ((decimal) bestBidD)

There just looks like there is a bit of implicit rounding happening here. :-\

tompazourek commented

First of all, if you use the tolerance of 0.000000000001, the "Best Ask" 0.0001717 gets parsed as 152/885265 (which is approximately 0.0001716999994), not as 1717/10000000. If you want it to be parsed as 1717/10000000, don't use the tolerance or set it to 0.

However, I think your issue is actually happening during formatting. You're using the %f placeholder, which is for floats. For decimals, you should be using the %M formatting placeholder.

NullVoxPopuli commented

Hey that was it! changing "%f" to "%M" fixed it.
(I'm newish to F#, sorry!)

Thanks!

(But also, I was just pasting what the printfn gave me, I wasn't trying to say the fractions were wrong or anything)

tompazourek commented

No problem, glad to have helped you! :-)

  • 👍1