Hello, how can I change the precision of a numeric? E.g. when I've got a numeric with 10,000 digits and then want that numeric truncated to 10 digits. I can't find anything in your tutorial, your reference nor via google. Thanks in advance, Timo
Timo Schlüßler wrote:
how can I change the precision of a numeric? E.g. when I've got a numeric with 10,000 digits and then want that numeric truncated to 10 digits.
Try constructing a numeric representing 1.0 to 10 Digits and multiply your number with that. -richy. -- Richard B. Kreckel <http://www.ginac.de/~kreckel/>
Richard B. Kreckel wrote:
Timo Schlüßler wrote:
how can I change the precision of a numeric? E.g. when I've got a numeric with 10,000 digits and then want that numeric truncated to 10 digits.
Try constructing a numeric representing 1.0 to 10 Digits and multiply your number with that.
-richy.
Thanks, that works, but isn't there a method/another way for truncating? When setting "Digits = 2" for the 1.0 numeric, the result (of multiply) stil has 4 decimal places ...
Timo Schlüßler wrote:
When setting "Digits = 2" for the 1.0 numeric, the result (of multiply) stil has 4 decimal places ...
Actually, it has even more decimal digits but the printing routine truncates fractional trailing zeros. The CLN preprocessor macro intDsize determines the granularity of binary digits. E.g. on x86, the precision is always a multiple of 32 bit. On amd64, it is a multiple of 64 bit. -richy. -- Richard B. Kreckel <http://www.ginac.de/~kreckel/>
Richard B. Kreckel wrote:
Timo Schlüßler wrote:
When setting "Digits = 2" for the 1.0 numeric, the result (of multiply) stil has 4 decimal places ...
Actually, it has even more decimal digits but the printing routine truncates fractional trailing zeros.
The CLN preprocessor macro intDsize determines the granularity of binary digits. E.g. on x86, the precision is always a multiple of 32 bit. On amd64, it is a multiple of 64 bit.
-richy. Okay ... thank you.
Timo
participants (2)
-
Richard B. Kreckel
-
Timo Schlüßler