Counting things is 80% of data work
Two more small tools complete your pipeline kit:
sort: prints lines in order.-nsorts numerically (so 9 comes before 10),-rreverses.uniq: collapses adjacent duplicate lines into one.uniq -calso prefixes each line with its count.
The classic gotcha: uniq only sees duplicates that are next to each other. That's why almost every use of uniq is sort file | uniq. Sort groups the duplicates together, then uniq collapses them.
The legendary combo sort | uniq -c | sort -rn answers "what are the most common lines?" and shows up everywhere: top visitors in a web log, most frequent error, most-used command in your history.
Code exercise · bash
Run it. votes.txt holds 6 votes for 3 fruits. sort groups them, and sort | uniq gives the deduplicated ballot.
Code exercise · bash
Find the winner: uniq -c counts each fruit, sort -rn puts the biggest count first, head -n 1 keeps the winner. (xargs at the end just trims uniq's decorative spacing so the output is clean.)
cut: picking columns
Much real data is line-based and column-based: CSV exports, server logs with fields, /etc/passwd. cut extracts columns from each line: -d, sets the delimiter (the character that separates fields — here a comma) and -f1 picks field 1. So cut -d, -f1 people.csv prints just the first column.
It slots straight into pipelines with everything above: cut -d, -f3 log.csv | sort | uniq -c | sort -rn answers "what's the most common value in column 3?" in one line — a question that would otherwise mean opening a spreadsheet.
Code exercise · bash
Run it. The first cut pulls the name column out of a 3-line CSV. The second feeds the city column through sort to find the alphabetically first city.
Quiz
Why does `uniq` usually need `sort` in front of it?
Code exercise · bash
Your turn. The starter writes 5 color votes (with repeats). Print the deduplicated sorted colors, then print how many DIFFERENT colors there are. Expected output: ``` blue green red 3 ```