Gawk/Settingprecision
Next: Setting the rounding mode, Previous: Try To Round, Up: FP Math Caution [Contents][Index]
16.4.4 Setting the Precision
gawk
uses a global working precision; it does not keep track of
the precision or accuracy of individual numbers. Performing an arithmetic
operation or calling a builtin function rounds the result to the current
working precision. The default working precision is 53 bits, which you can
modify using the predefined variable PREC
. You can also set the
value to one of the predefined caseinsensitive strings
shown in Table 16.4,
to emulate an IEEE 754 binary format.
PREC

IEEE 754 binary format 

"half"

16bit halfprecision 
"single"

Basic 32bit single precision 
"double"

Basic 64bit double precision 
"quad"

Basic 128bit quadruple precision 
"oct"

256bit octuple precision 
The following example illustrates the effects of changing precision on arithmetic operations:
$ gawk M v PREC=100 'BEGIN { x = 1.0e400; print x + 0 > PREC = "double"; print x + 0 }'  1e400  0
CAUTION: Be wary of floatingpoint constants! When reading a floatingpoint
constant from program source code,
gawk
uses the default precision (that of a Cdouble
), unless overridden by an assignment to the special variablePREC
on the command line, to store it internally as an MPFR number. Changing the precision usingPREC
in the program text does not change the precision of a constant.If you need to represent a floatingpoint constant at a higher precision than the default and cannot use a commandline assignment to
PREC
, you should either specify the constant as a string, or as a rational number, whenever possible. The following example illustrates the differences among various ways to print a floatingpoint constant:$ gawk M 'BEGIN { PREC = 113; printf("%0.25f\n", 0.1) }'  0.1000000000000000055511151 $ gawk M v PREC=113 'BEGIN { printf("%0.25f\n", 0.1) }'  0.1000000000000000000000000 $ gawk M 'BEGIN { PREC = 113; printf("%0.25f\n", "0.1") }'  0.1000000000000000000000000 $ gawk M 'BEGIN { PREC = 113; printf("%0.25f\n", 1/10) }'  0.1000000000000000000000000
Next: Setting the rounding mode, Previous: Try To Round, Up: FP Math Caution [Contents][Index]