Synergex/HarmonyCore

16 digit Decimal value is being displayed as a rounded value.

tscaff opened this issue · 1 comments

We noticed during testing on a Harmony Core routine that returns a 16 digit decimal with the value 9,999,999,999,999,999 that Swagger is displaying a rounded value.

value read on disk:
9999999999999999

value from swagger:
10000000000000000

I tried check JSON documentation to see if we are at some sort of upper limit on our number, but I am not finding anything that would explain this behavior.

The value is not being rounded prior to return so we are a little perplexed on why this is occurring. Is this a Swagger display issue?

Javascript and by extension the swagger client has limitations around numbers, basically everything is internally stored as a floating point number. As you get a number large enough things get pretty imprecise. I think if you look at the actual data sent during the request, it will have the correct non rounded number. I think there are some arbitrary precision number stacks written in javascript to help work around this but i don't really have any direct experience. Depending on the implementation details of your number library, you may need to tag this field as HARMONY_AS_STRING.