Gawk/Setting-precision

16.4.4 Setting the Precision

gawk uses a global working precision; it does not keep track of the precision or accuracy of individual numbers. Performing an arithmetic operation or calling a built-in function rounds the result to the current working precision. The default working precision is 53 bits, which you can modify using the predefined variable PREC. You can also set the value to one of the predefined case-insensitive strings shown in Table 16.4, to emulate an IEEE 754 binary format.

`PREC`	IEEE 754 binary format
`"half"`	16-bit half-precision
`"single"`	Basic 32-bit single precision
`"double"`	Basic 64-bit double precision
`"quad"`	Basic 128-bit quadruple precision
`"oct"`	256-bit octuple precision

Table 16.4: Predefined precision strings for PREC

The following example illustrates the effects of changing precision on arithmetic operations:

$ gawk -M -v PREC=100 'BEGIN { x = 1.0e-400; print x + 0
>   PREC = "double"; print x + 0 }'
-| 1e-400
-| 0

CAUTION: Be wary of floating-point constants! When reading a floating-point
constant from program source code, gawk uses the default precision (that of a C double), unless overridden by an assignment to the special variable PREC on the command line, to store it internally as an MPFR number. Changing the precision using PREC in the program text does not change the precision of a constant.
If you need to represent a floating-point constant at a higher precision than the default and cannot use a command-line assignment to PREC, you should either specify the constant as a string, or as a rational number, whenever possible. The following example illustrates the differences among various ways to print a floating-point constant:
$ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 0.1) }'
-| 0.1000000000000000055511151
$ gawk -M -v PREC=113 'BEGIN { printf("%0.25f\n", 0.1) }'
-| 0.1000000000000000000000000
$ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", "0.1") }'
-| 0.1000000000000000000000000
$ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 1/10) }'
-| 0.1000000000000000000000000