Normalize line endings before a text comparison

By Batu · English technical notes

Also published in our Blogger archive.

Two text values may represent the same lines while differing byte-for-byte because one uses Windows CRLF line endings, another uses classic CR endings, and another uses LF. When the comparison rule considers those forms equivalent, normalize each value first. This example replaces CRLF with LF before replacing remaining CR characters. That order matters: replacing CR first would turn a CRLF pair into two LF characters.

The left value contains all three common forms. After normalization it equals the right value, whose lines use LF. A second assertion verifies that applying the function to already-normalized text leaves this particular input unchanged. The output uses repr so the final newline and embedded newline characters are visible rather than rendered as line breaks.

This function intentionally changes only newline representation. It does not remove a final newline, trim spaces, repair text decoding, or make all Unicode-equivalent strings compare equal. If those distinctions matter, specify them separately; normalizing more than the comparison contract requires can hide a real difference. Python string replace is available in all supported modern Python versions, while the f-string used for the first output line requires Python 3.6+.

Reference: Python str.replace documentation.

AI-assistance disclosure: This article was drafted with AI assistance and should be reviewed in its application context.

def normalize_line_endings(text):
    return text.replace("\r\n", "\n").replace("\r", "\n")

left = "alpha\r\nbeta\rgamma\n"
right = "alpha\nbeta\ngamma\n"
normalized = normalize_line_endings(left)

assert normalized == right
assert normalize_line_endings(right) == right

print(f"equal={normalized == right}")
print(repr(normalized))

equal=True
'alpha\nbeta\ngamma\n'