Detect an unexpected CSV column without rejecting useful exports

By Batu · English technical notes

Also published in our Blogger archive.

Quick answer

Header labels are ordinary CSV fields, so an allowlist can compare them without reading or rewriting any data records.

Example

The example computes headers outside the allowed set and serializes only debug_flag as an unexpected field. It does not delete the column or reject rows; it preserves evidence that an export shape changed.

import csv, io, json
headers = next(csv.reader(io.StringIO('id,name,debug_flag\n')))
allowed = {'id', 'name'}
unexpected = [h for h in headers if h not in allowed]
assert unexpected == ['debug_flag']
print(json.dumps({'unexpected': unexpected}, separators=(',', ':')))

Expected stdout:

{"unexpected":["debug_flag"]}

Reading the result

A required-column check is complementary. An allowlist finding cannot prove that all fields the importer needs are present.

Include the expected allowlist version in a review report. That makes it clear whether debug_flag is a newly observed export field or a label intentionally ignored by an older mapping.

Preserve header order in a fuller report if a downstream mapper uses positions. The simple allowlist here answers only whether a label is outside the accepted set.

Sources

- Python csv module documentation

Prepared with AI assistance. The example uses synthetic data; its stated limits apply.