Setting precision (The GNU Awk User’s Guide)
Next: Setting the rounding mode, Previous: Try To Round, Up: FP Math Caution [Contents][Index]
16.4.4 Setting the Precision
gawk
uses a global working precision; it does not keep track of the precision or accuracy of individual numbers. Performing an arithmetic operation or calling a built-in function rounds the result to the current working precision. The default working precision is 53 bits, which you can modify using the predefined variable PREC
. You can also set the value to one of the predefined case-insensitive strings shown in Table 16.4, to emulate an IEEE 754 binary format.
PREC
|
IEEE 754 binary format |
---|---|
"half"
|
16-bit half-precision |
"single"
|
Basic 32-bit single precision |
"double"
|
Basic 64-bit double precision |
"quad"
|
Basic 128-bit quadruple precision |
"oct"
|
256-bit octuple precision |
The following example illustrates the effects of changing precision on arithmetic operations:
$ gawk -M -v PREC=100 'BEGIN { x = 1.0e-400; print x + 0 > PREC = "double"; print x + 0 }' -| 1e-400 -| 0
CAUTION: Be wary of floating-point constants! When reading a floating-point constant from program source code,
gawk
uses the default precision (that of a Cdouble
), unless overridden by an assignment to the special variablePREC
on the command line, to store it internally as an MPFR number. Changing the precision usingPREC
in the program text does not change the precision of a constant.If you need to represent a floating-point constant at a higher precision than the default and cannot use a command-line assignment to
PREC
, you should either specify the constant as a string, or as a rational number, whenever possible. The following example illustrates the differences among various ways to print a floating-point constant:$ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 0.1) }' -| 0.1000000000000000055511151 $ gawk -M -v PREC=113 'BEGIN { printf("%0.25f\n", 0.1) }' -| 0.1000000000000000000000000 $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", "0.1") }' -| 0.1000000000000000000000000 $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 1/10) }' -| 0.1000000000000000000000000
Next: Setting the rounding mode, Previous: Try To Round, Up: FP Math Caution [Contents][Index]