Decode HTML character references safely

By Batu · English technical notes

Also published in our Blogger archive.

HTML text often contains named references such as & and £, plus decimal or hexadecimal numeric references. html.unescape converts those references into Unicode characters using HTML 5 rules. In this example, it changes a pound reference, an ampersand reference, and the hexadecimal reference for a smiling face. The assertions verify the whole decoded result and a separate angle-bracket conversion before the program prints the exact string.

This is a useful operation when the input is text known to contain HTML character references and the next step needs the readable characters. It is deterministic for the shown input and requires no network access or parser configuration. html.unescape was added in Python 3.4.

“Safely” here means using the standard library’s defined HTML 5 reference-handling rules, not that the result is safe to render as HTML. Decoding <tag> produces <tag>; if that text will be inserted into an HTML document, apply context-appropriate escaping at the output boundary. The function is also not an HTML parser, sanitizer, or validator, and it does not establish that a larger document is trustworthy. Preserve the original text too when an application needs auditability or round-trip fidelity.

Reference: Python html.unescape documentation.

AI-assistance disclosure: This article was drafted with AI assistance and should be reviewed in its application context.

from html import unescape

encoded = "Cost: &pound;5 &amp; tax; smile: &#x1F642;."
decoded = unescape(encoded)

assert decoded == "Cost: £5 & tax; smile: 🙂."
assert unescape("&lt;tag&gt;") == "<tag>"

print(decoded)

Cost: £5 & tax; smile: 🙂.