What happens when you type a URL

The whole journey in six steps

You type https://example.com/about in a browser and press Enter. Here is everything that happens, in order. This list is the map for the entire course.

1. Parse the URL. The browser splits the address into pieces: which protocol, which server, which page.
2. DNS lookup. The browser asks the DNS system to turn the name example.com into an IP address, something like 93.184.215.14. (Unit 2)
3. TCP connection. The browser opens a reliable connection to that IP address. (Unit 3)
4. TLS handshake. Because the URL starts with https, the browser and server agree on encryption keys so nobody can read the traffic. (Unit 7)
5. HTTP request and response. The browser sends GET /about, the server sends back the page. (Units 4 to 6)
6. Render. The browser turns the response into pixels, then repeats steps 2 to 5 for every image, stylesheet, and script the page needs.

All of this usually finishes in well under a second.

What happens first

The first thing that happens is that the browser looks up the IP address for example.com.

The browser cannot send anything to the server until it knows the server's address, so the DNS lookup has to come before any HTTP request. There is nowhere to send the request until the name has been resolved.

The full order is: parse the URL, DNS, connect, encrypt, request, response, render.

Worth noticing that the ordering is forced rather than chosen. Each step needs the output of the one before it, which is also why a failure early in the chain looks like a failure of everything after it. A broken DNS lookup and a broken server both present as a page that does not load.

Anatomy of a URL

A URL (Uniform Resource Locator) is a structured address. Every part tells the browser something specific:

https://shop.example.com:443/cart/items?color=blue
└─┬─┘   └──────┬───────┘└┬┘└────┬────┘ └───┬────┘
scheme        host      port   path      query

• scheme: which protocol to speak. https means HTTP with encryption.
• host: which server, by name. DNS will turn this into an IP address.
• port: a number that selects which program on that server should receive the connection, because one machine runs many network programs at once and the port tells them apart. Unit 2 covers this. It is usually omitted, since https implies 443 and http implies 80.
• path: which resource on that server you want.
• query: extra key=value details after the ?, joined by &.

Because a URL is just text with separators, you can pull it apart with any programming language. Let's do it in bash, which you already know from the terminal.

Splitting a URL into its parts

Bash's built-in text trimming is enough to pull a URL apart. ${url%%://*} keeps everything before the first ://, and ${url#*://} removes it.

url="https://shop.example.com/cart/items?color=blue"

scheme=${url%%://*}
rest=${url#*://}
host=${rest%%/*}
target=/${rest#*/}

echo "scheme: $scheme"
echo "host: $host"
echo "target: $target"

Output

scheme: https
host: shop.example.com
target: /cart/items?color=blue

The two trimming forms cut from opposite ends. %% removes a match from the right, and # removes one from the left, which is how four lines separate three fields.

Note the leading / put back manually on the target line. Trimming up to the first / consumed it, so the code re-adds it, which is the sort of small detail that turns into a bug when a path silently loses its opening slash.

Splitting the target into path and query

Two more trims separate the path from the query string at the ?.

url="http://api.example.com/users?id=42&page=2"

rest=${url#*://}
host=${rest%%/*}
target=/${rest#*/}

path=${target%%\?*}
query=${target#*\?}

echo "host: $host"
echo "path: $path"
echo "query: $query"

Output

host: api.example.com
path: /users
query: id=42&page=2

Reading the trims

• The ? must be escaped as \? in the pattern, because bash otherwise treats it as a wildcard matching any single character.
• path=${target%%\?*} trims from the first ? to the end, and query=${target#*\?} trims everything up to and including the first ?. The same pair of operators as before, aimed at a different separator.
• The query keeps its & separators intact, since splitting id=42&page=2 into individual keys and values would take another loop. Servers do that work for you, which is why you rarely parse a query string by hand.