Check whether CSV header normalization creates a collision
Also published in our Blogger archive.
Quick answer
Header normalization is application policy; csv.reader preserves raw strings and does not decide whether labels are equivalent.
Example
The code trims and case-folds into a second list, then looks for repeated normalized values. Name and name remain distinct raw evidence while both produce the name collision finding.
import csv, io
headers = next(csv.reader(io.StringIO('Name, name ,ID\n')))
normalized = [h.strip().casefold() for h in headers]
collisions = sorted({x for x in normalized if normalized.count(x) > 1})
assert collisions == ['name']
print('collisions:', collisions)Expected stdout:
collisions: ['name']Reading the result
Unicode normalization and punctuation removal can create further collisions. Add them only when the mapping contract requires them and show the raw-to-normalized mapping in review output.
Run this check before constructing a normalized dictionary. A dictionary lookup after collision has already lost the evidence needed to explain why an apparent field match was ambiguous.
The proposed normalized labels are computed in a separate collection. That is what makes it possible to reject the ambiguous transformation without losing raw input.
Sources
- Python csv module documentation
Prepared with AI assistance. The example uses synthetic data; its stated limits apply.