Query parameters and URL encoding

Passing data in the URL

You met the query string in lesson 1-2: everything after the ?, as key=value pairs joined by &.

curl "https://api.example.com/search?q=coffee&page=2"

(Quote the URL in a shell, otherwise the shell treats & as its own operator.)

One complication: URLs only allow a limited set of characters. Spaces, accents, and symbols like & inside a value would break the structure. The fix is percent-encoding: each forbidden byte becomes % plus its hex value. A space becomes %20 (or + in query strings), é becomes %C3%A9, and a literal & in a value becomes %26.

Encoding bugs are a classic source of quiet failures: an unencoded & inside a search term silently splits it into two parameters, and the API "works" while filtering on garbage. So the rule is: you never encode by hand. Every language has a helper, and it is a one-liner in Python:

Encoding parameters correctly

urlencode takes a dict of parameters and produces a correctly encoded query string.

from urllib.parse import urlencode

params = {"q": "coffee shops", "city": "sao paulo", "page": 2}
url = "https://api.example.com/search?" + urlencode(params)

print(url)

Output

https://api.example.com/search?q=coffee+shops&city=sao+paulo&page=2

The spaces in the values became +, which is the query-string shorthand for a space. Outside a query string, in a path, the same space would be written %20.

The helper also inserts the & between pairs and the = inside them, so the structural characters are added by the encoder while the value characters are escaped by it. Doing both jobs in one place is what makes it safe.

Notice that page was the integer 2 and came out as text. Everything in a URL is characters, so the encoder converts as it goes, and the server will parse "2" back into a number if it wants one.

Parsing a query string as the server does

The reverse direction: split on & for the pairs, then on = for each key and value.

query = "id=42&sort=newest&debug=true"

for pair in query.split("&"):
    key, value = pair.split("=")
    print(key, "->", value)

Output

id -> 42
sort -> newest
debug -> true

Reading the code

• query.split("&") gives the list ['id=42', 'sort=newest', 'debug=true'], one string per parameter.
• key, value = pair.split("=") unpacks each pair in one line, which works because the split produces exactly two items.
• print(key, "->", value) adds the spaces between arguments for you, so no manual joining is needed.
• That two-item assumption is exactly why encoding matters. A value containing an unencoded = would split into three parts and raise an error, and a value containing an unencoded & would silently become two parameters, which is the quiet failure from the previous block.

Decoding %20

The file is actually called annual report.pdf, because %20 is a percent-encoded space.

Hex 20 is decimal 32, which is the space character, so the encoding writes the byte value where the byte itself is not allowed. The server decodes it back before looking up the file.

When a URL looks garbled with % signs, decoding it mentally is usually enough. Most of what you will see is spaces and ordinary punctuation, and %2F for a slash inside a value is the one that most often causes real confusion.

A useful check while debugging: a filename that works in one place and 404s in another is often an encoding mismatch, where one side encoded the space and the other sent it raw.