cqframework/cql-execution

Issues w/ Power operator and large decimals

Opened this issue · 2 comments

cmoesel commented

See spec-tests:

  • ValueLiteralsAndSelectors.Decimal.Decimal10Pow28ToZeroOneStepDecimalMaxValue
  • ValueLiteralsAndSelectors.Decimal.DecimalPos10Pow28ToZeroOneStepDecimalMaxValue
  • ValueLiteralsAndSelectors.Decimal.DecimalNeg10Pow28ToZeroOneStepDecimalMinValue

These should return large numbers but are returning null. Is the number getting switched to null because it thinks its an integer and overflowed?

kjmahalingam commented

This is directly tied to #204 since the assumption above is correct. If we take ValueLiteralsAndSelectors.Decimal.Decimal10Pow28ToZeroOneStepDecimalMaxValue as an example, the output value here:

<output>10*10000000000000000000.00000000-0.00000001</output>

has the first argument to the Subtract parsed as null since the 1e28 is processed as an Integer that is considered to overflow. In order to fix this, we need to fix #204.

kjmahalingam commented

In addition, we need to update the values of both the expressions and outputs in these tests. They are set to be 10^28 as opposed to (10^28 - 1) / (10^8) as is defined to be the max in the CQL spec.