Gawk/Full-Line-Fields

4.5.5 Making the Full Line Be a Single Field

Occasionally, it’s useful to treat the whole input line as a single field. This can be done easily and portably simply by setting FS to "\n" (a newline):²²

awk -F'\n' 'program' files …

When you do this, $1 is the same as $0.

Changing FS Does Not Affect the Fields

According to the POSIX standard, awk is supposed to behave as if each record is split into fields at the time it is read. In particular, this means that if you change the value of FS after a record is read, the values of the fields (i.e., how they were split) should reflect the old value of FS, not the new one.

However, many older implementations of awk do not work this way. Instead, they defer splitting the fields until a field is actually referenced. The fields are split using the current value of FS! (d.c.) This behavior can be difficult to diagnose. The following example illustrates the difference between the two methods:

sed 1q /etc/passwd | awk '{ FS = ":" ; print $1 }'

which usually prints:

root

on an incorrect implementation of awk, while gawk prints the full first line of the file, something like:

root:x:0:0:Root:/:

(The sed²³ command prints just the first line of /etc/passwd.)

Footnotes

(22)

Thanks to Andrew Schorr for this tip.

(23)

The sed utility is a “stream editor.” Its behavior is also defined by the POSIX standard.