Limited Decimal-Value in number()-function

[webnic]

Hi Team,

first of all, great work. I love your approach. It's that simple, even I can use it.. ;)

However, since I need to transform Json2Json containing large decimal-values consumed as String, I noticed a significant limitation in the number()-function.
Obviously I have a parsing-limitation for the Integer (Max-Int) and a max precision of Double. The return-type is also a DoubleNode: com.schibsted.spt.data.jslt.impl.NodeUtils:172-194.

Is it an option to extend the existing number()-function or create a new function for precise Decimal-Values? Parsing into a BigDecimal an return a DecimalNode or something like that..

Thx and regards

[larsga]

I think probably the best way to deal with this is to give number() support for large decimals by returning BigDecimal or similar. Jackson supports it, so there's no real reason why we shouldn't be able to support this.

Could you give us some examples of the decimal values?

[webnic]

Sounds great! These are some samples from recent messages:
-"bid": "11234138.23975"
-"ask": "-72915539138.234933"
-"PV": "17236399.795802329"

[larsga]

Two of these were already working. Commit 08fbaef fixes so the third one also works now.

But perhaps you have even bigger examples?

[webnic]

For these numbers I receive currently these results:
11234138.23975 => 1.123413823975E7 => scientific notation
-72915539138.234933 => Exception
17236399.795802329 => 1.723639979580233E7 => scientific notation and rounded

[webnic]

I see, your fix should fix the Exception for the 2nd example. Thank you for that.

[webnic]

With your fix I would get -7.291553913823494E10 scientific and rounded up

[larsga]

The scientific notation is not a JSLT issue, because JSLT will return a double to you. So whether it's scientific notation or not will depend on how you serialize the result. You can configure Jackson not to use scientific notation if you want, but you shouldn't really care. It's the same numbers either way.

Any rounding is a different issue, of course.

Looks like -72915539138.234933 just isn't expressible as a Java double. It ends up rounded off to -72915539138.23494-72915539138.23494. So to solve this one we really would need to use a DecimalNode containing a BigDecimal. I just checked, and it did work.

[webnic]

In my opinion that is a part of the problem. Wouldn’t it be easier just using a BigDecimal for Decimal-Values?
I barely remember my datastructure lectures with all the mantissa, fraction and precision stuff. So I have forgotten why. However, while debugging, I see that I am getting a scientific notation when I have a intPart with more than 7 digits. The number is obviously too big for double representation. But an additional problem with Jackson is quite possible.