Sort casefold-colliding Unicode labels with a deterministic tie breaker

By Batu · English technical notes

Sort casefold-colliding Unicode labels with a deterministic tie breaker

For a repeatable, locale-independent order, sort by a tuple—but choose its second component for the policy you need. key=str.casefold alone does not create a complete order for Straße, STRASSE, and Strasse: all fold to strasse. Python sorting is stable, so equal keys retain source order. The first two output lines demonstrate that two incoming record orders therefore produce different results.

A display-oriented policy can use (label.casefold(), label). It gives the requested spelling order STRASSE, Strasse, Straße, and retains all three records. That is useful when the displayed spelling itself should resolve ties. It is not the only deterministic policy, however. The final line uses (label.casefold(), record_id). Its order differs because stable IDs, rather than Unicode code-point ordering of display text, decide among equivalent folded labels.

For records that must retain a persistent identity across display-label edits, a stored ID is usually the clearer tie breaker. For a user-facing alphabetical listing, the label may instead be intentional. The important point is to make the secondary rule explicit; neither approach deduplicates casefold-equivalent labels. str.casefold() is available in Python 3.3+ and is intended for caseless matching. This example uses Python's string ordering, not linguistic collation. The str.casefold() documentation describes caseless matching, and the sorting HOWTO documents stable sorting.

AI assistance disclosure: This article was prepared with AI assistance and checked with the synthetic assertions shown below.

records_one = [
    {"record_id": 20, "label": "Straße"},
    {"record_id": 30, "label": "STRASSE"},
    {"record_id": 10, "label": "Strasse"},
]
records_two = list(reversed(records_one))

casefold_one = sorted(records_one, key=lambda record: record["label"].casefold())
casefold_two = sorted(records_two, key=lambda record: record["label"].casefold())
display_order = sorted(
    records_one, key=lambda record: (record["label"].casefold(), record["label"])
)
id_order = sorted(
    records_one, key=lambda record: (record["label"].casefold(), record["record_id"])
)

labels = [record["label"] for record in records_one]
assert [label.casefold() for label in labels] == ["strasse"] * 3
assert [record["record_id"] for record in casefold_one] == [20, 30, 10]
assert [record["record_id"] for record in casefold_two] == [10, 30, 20]
assert [record["label"] for record in display_order] == ["STRASSE", "Strasse", "Straße"]
assert [record["record_id"] for record in id_order] == [10, 20, 30]

print("casefold only, source one:", [record["record_id"] for record in casefold_one])
print("casefold only, source two:", [record["record_id"] for record in casefold_two])
print("display-text tie breaker:", [record["label"] for record in display_order])
print("stable-ID tie breaker:", [record["record_id"] for record in id_order])

casefold only, source one: [20, 30, 10]
casefold only, source two: [10, 30, 20]
display-text tie breaker: ['STRASSE', 'Strasse', 'Straße']
stable-ID tie breaker: [10, 20, 30]