π§ awk β The Swiss Army knife of Linux text processing
Most people use grep to find text.
Most people use cut to slice columns.
Most people do not know awk can do both plus math plus conditions plus formatting in a single line.
Here is everything you need to know.
----------------
π€ WHAT IS awk?
----------------
awk is a pattern scanning and text processing language built into every Linux system.
It reads a file line by line, splits each line into fields, and lets you filter, transform, calculate, and format the output however you want.
One tool. Infinite use cases.
----------------
π§± BASIC SYNTAX
----------------
awk 'pattern { action }' filename
β’ pattern β which lines to process. Leave empty to process all lines.
β’ action β what to do with those lines.
β’ $0 β the entire line.
β’ $1, $2, $3 β individual fields split by whitespace by default.
β’ NR β current line number.
β’ NF β total number of fields in the current line.
----------------
βοΈ EVERYDAY EXAMPLES
----------------
β’ Print the second column of every line
β awk '{ print $2 }' file.txt
β’ Print line numbers alongside content
β awk '{ print NR, $0 }' file.txt
β’ Print only lines where third column is greater than 100
β awk '$3 > 100' file.txt
β’ Print the last field of every line
β awk '{ print
$NF }' file.txt
β’ Print lines containing the word error
β awk '/error/ { print }' file.txt
β’ Count total number of lines
β awk 'END { print NR }' file.txt
β’ Sum all values in the first column
β awk '{ sum = $1 } END { print sum }' file.txt
β’ Print lines between two patterns
β awk '/START/,/END/' file.txt
----------------
π§ FIELD SEPARATOR
----------------
By default awk splits on whitespace. Change this with -F.
β’ Parse a CSV file
β awk -F',' '{ print $1, $3 }' file.csv
β’ Parse /etc/passwd and print usernames and shells
β awk -F':' '{ print $1,
$NF }' /etc/passwd
β’ Print the third field from a colon separated file
β awk -F':' '{ print $3 }' file.txt
----------------
π‘ REAL WORLD ANALOGY
----------------
Think of awk like a spreadsheet formula engine for your terminal.
Excel lets you pick columns, filter rows, and calculate totals through a GUI.
awk does the exact same thing on any text file directly from the command line.
No GUI. No import. Just one line and you have your answer.
----------------
π₯ REAL WORLD USE CASES
----------------
β’ Top 5 processes consuming the most memory
β ps aux | awk '{ print $4, $11 }' | sort -rn | head -5
β’ Calculate total disk usage from du output
β du -sh * | awk '{ sum = $1 } END { print sum }'
β’ Extract failed SSH login IPs from auth log
β awk '/Failed password/ { print $11 }' /var/log/auth.log | sort | uniq -c | sort -rn
β’ Print lines where response time exceeds 500ms
β awk '$7 > 500 { print $0 }' access.log
β’ Remove duplicate lines while preserving order
β awk '!seen[$0] ' file.txt
β’ Print header line plus any line matching a pattern
β awk 'NR==1 || /error/' file.txt
----------------
β‘ awk vs grep vs cut vs sed
----------------
β’ grep β find lines matching a pattern. That is all it does well.
β’ cut β extract fixed columns by delimiter. No logic, no math, no conditions.
β’ sed β stream editor for substitution and deletion. Great for find and replace.
β’ awk β does everything above plus arithmetic, conditionals, variables, loops, and formatted output.
When one tool can replace three, use that tool.
----------------
π¬ Golden rule:
If you are piping grep into cut into another grep, stop. You probably just need one awk command.
π Save this for the next time you are staring at a log file wondering how to extract exactly what you need.
What is your most used awk one-liner? Drop it below π
#Linux #awk #CommandLine #DevOps #SysAdmin #ShellScripting #LinuxTips #Terminal #LearnLinux #SRE