Mastering AWK and NF: The Practical Guide to Seamless Data Processing

Introduction

When you start learning text processing on the command line, many people encounter awk.

Awk is a tool capable of powerful data processing with simple syntax, but there are quite a few points where beginners tend to stumble.

One of them is understanding the special variable NF. NF may seem modest at first glance, but it plays a very important role in log analysis and data formatting in practical work.

This article carefully explains NF in awk from the basics to advanced usage, from the perspective of a beginner.

Rather than simply explaining commands, it is structured with an awareness of why that usage is the way it is and where people tend to get stuck. Let's deepen our understanding by following along with the flow.

Reference: GNU awk

Basic Definition and Role of the Special Variable “NF” in awk

Creating the File

cat << 'EOF' > input.txt
apple orange banana
cat dog
red blue green yellow
EOF

Command to Run

awk '{print NF}' input.txt

Output

3
2
4

Command to Run

awk '{print $NF}' input.txt

Output

banana
dog
yellow

Command to Run

awk '{print $1, $NF}' input.txt

Output

apple banana
cat dog
red yellow

How It Works

Item	Description
NF	A special variable that represents the number of fields (columns) in a record (one line)
$NF	Points to the last field (the NF-th field)
$1	The first field
awk behavior	Processes line by line and automatically splits fields (default delimiter is whitespace)
Use cases	Checking column count, extracting the last column, processing variable-length data

Explanation

NF automatically holds the number of fields for each line, making it very convenient for processing variable-column data.
By using $NF, you can always retrieve the "last value" even when the number of columns differs.

How to Distinguish Between Referencing Field Count and Extracting the Last Column

Creating the File

cat << 'EOF' > input.txt
apple orange banana
cat dog
red blue green yellow
one
EOF

Command to Run

awk '{print NF}' input.txt

Output

3
2
4
1

Command to Run

awk '{print $NF}' input.txt

Output

banana
dog
yellow
one

How It Works

Item	Description	Example ("apple orange banana")
NF	Built-in variable representing the number of fields (columns)	3
$NF	Retrieves the last field	banana
$1～$n	References each field by number	$1=apple, $2=orange

Explanation

NF returns the number of columns per line, and $NF uses that column count as an index to retrieve the last column.
For data with variable column counts, you can process flexibly by using these two in combination.

How to Extract and Filter Only Rows with a Specific Number of Columns

Creating the File

cat << 'EOF' > input.txt
a b c
1 2
x y z w
hello world
EOF

Command to Run

awk 'NF==3' input.txt

Output

a b c

Command to Run

awk 'NF==2' input.txt

Output

1 2
hello world

How It Works

Element	Description
awk	Text processing tool
NF	Built-in variable representing the number of fields (columns)
NF==3	Extracts only rows with 3 columns
NF==2	Extracts only rows with 2 columns
Delimiter	Default is whitespace (spaces and tabs)

Explanation

Because NF automatically counts the number of columns for each line, you can easily filter just by specifying a condition.
This is a basic technique commonly used in log formatting and invalid data detection.

Combining NF with Loop Processing (for Statement) to Dynamically Operate on All Columns

Creating the File

cat << 'EOF' > input.txt
apple orange banana
cat dog
red blue green yellow
EOF

Command to Run

awk '{print NF}' input.txt

Output

3
2
4

Command to Run

awk '{
  for(i=1; i<=NF; i++){
    print "Column" i ":" $i
  }
}' input.txt

Output

Column1:apple
Column2:orange
Column3:banana
Column1:cat
Column2:dog
Column1:red
Column2:blue
Column3:green
Column4:yellow

How It Works

Element	Description
NF	Holds the number of fields (columns) in the current line
$i	References the i-th field
for(i=1; i<=NF; i++)	Loops from the 1st column to the last column
print	Dynamically outputs each field

Explanation

Using NF allows flexible processing even for data with different numbers of columns.
The key point is that combining it with a for statement lets you automatically scan through all columns.

The Effect of Changing the Delimiter (FS) on the NF Value and Points to Watch

Creating the File

cat << 'EOF' > input.txt
apple orange banana
dog,cat,bird
one:two:three:four
EOF

Command to Run

awk '{print $0 " -> NF=" NF}' input.txt

Output

apple orange banana -> NF=3
dog,cat,bird -> NF=1
one:two:three:four -> NF=1

Command to Run

awk -F',' '{print $0 " -> NF=" NF}' input.txt

Output

apple orange banana -> NF=1
dog,cat,bird -> NF=3
one:two:three:four -> NF=1

Command to Run

awk -F':' '{print $0 " -> NF=" NF}' input.txt

Output

apple orange banana -> NF=1
dog,cat,bird -> NF=1
one:two:three:four -> NF=4

How It Works

FS (Delimiter)	Target Line	Split Result	NF
Default (whitespace)	apple orange banana	apple / orange / banana	3
Default (whitespace)	dog,cat,bird	Treated as 1 field	1
,	dog,cat,bird	dog / cat / bird	3
:	one:two:three:four	one / two / three / four	4
, or :	Lines with mismatched delimiter	Not split, remains 1 field	1

Explanation

Changing FS changes the splitting unit, so NF (the number of fields) also changes significantly.
Note that when the input data and FS do not match, NF will unintentionally become 1.

Techniques for Efficiently Analyzing Variable-Length Format Log Files with NF

Creating the File

cat << 'EOF' > input.txt
INFO 2026-05-01 userA login success
WARN 2026-05-01 userB disk almost_full
ERROR 2026-05-01 userC
DEBUG userD action=click button=submit extra=data
EOF

Command to Run

awk '{print $0, "=> NF=" NF}' input.txt

Output

INFO 2026-05-01 userA login success => NF=5
WARN 2026-05-01 userB disk almost_full => NF=5
ERROR 2026-05-01 userC => NF=3
DEBUG userD action=click button=submit extra=data => NF=5

Command to Run

awk 'NF < 5 {print "Missing fields:", $0}' input.txt

Output

Missing fields: ERROR 2026-05-01 userC

Command to Run

awk '{print "Last field:", $NF}' input.txt

Output

Last field: success
Last field: almost_full
Last field: userC
Last field: extra=data

How It Works

Concept	Description	Example
NF	Number of fields per line	NF=5
$NF	The last field	success
NF condition	判定 for variable-length data	NF < 5
Variable-length support	Can process even with different column counts	Error log extraction

Explanation

Using awk's NF allows flexible analysis even for logs with an undefined number of columns.
In particular, combining it with $NF lets you write last-field retrieval simply.

Validation Method Using NF with Conditional Branching (if Statement) to Exclude Incomplete Data Rows

Creating the File

cat << 'EOF' > input.txt
apple orange banana
grape
melon peach
kiwi
EOF

Command to Run

awk '{ if (NF >= 2) print }' input.txt

Output

apple orange banana
melon peach

How It Works

Element	Description
NF	Number of fields (column count separated by whitespace)
Conditional branching	if (NF >= 2) outputs only lines with 2 or more items
Exclusion targets	Incomplete lines with only 1 item (e.g., grape, kiwi)
Processing content	Only prints lines that satisfy the condition with an if statement

Explanation

Using an if statement makes the intent of the condition clear, resulting in more readable validation processing.
NF's strength is that it can make judgments quickly and efficiently.

How to Subtract a Specific Column Count from NF to Specify the “nth from the End”

Creating the File

cat << 'EOF' > input.txt
a b c d e
1 2 3 4 5
x y z
EOF

Command to Run

awk '{print $(NF)}' input.txt

Output

e
5
z

Command to Run

awk '{print $(NF-1)}' input.txt

Output

d
4
y

Command to Run

awk '{print $(NF-2)}' input.txt

Output

c
3
x

How It Works

Expression	Meaning	Example (a b c d e)	Output
NF	Total number of fields	5	e
NF-1	2nd from the end	5-1=4	d
NF-2	3rd from the end	5-2=3	c
$(NF-n)	(n+1)th from the end	variable	any

Explanation

NF represents the number of fields in a line, and by subtracting from it you can specify positions counting from the end.
Using the form $(NF-n), you can flexibly retrieve the column that is "n positions from the end."

Summary for Mastering awk and NF

NF in awk holds more meaning than just "number of columns."

It is a core concept in many situations, including retrieving the last column, checking data integrity, and handling variable-length data.

For beginners, the shortest path to understanding is to start with basic usage such as $NF and NF==number, then gradually combine them with for loops and conditional branching.