Disable SequenceMatcher autojunk for a repeated short alphabet
Disable SequenceMatcher autojunk for a repeated short alphabet
For a repeated short alphabet, construct difflib.SequenceMatcher with autojunk=False when you need repeated symbols to remain candidates for alignment. The automatic junk heuristic treats very frequent elements in a long second sequence as popular; that can be useful for ordinary text, but it can remove the only symbols available in synthetic or domain-specific repetitive data.
This experiment compares 240-element sequences: 120 "A" values followed by 120 "B" values, against the reverse ordering. With the default autojunk=True, both symbols are sufficiently frequent in the second sequence that the matcher reports only its required zero-size sentinel block. That does not mean the sequences share no values; it is a consequence of the heuristic. With autojunk=False, the matcher reports Match(a=0, b=120, size=120), aligning the first run of "A" values with the second run in the other sequence.
The assertion makes the differing block tables explicit. Disabling autojunk is not automatically more correct or faster: it can increase work and may choose a different equally plausible alignment when repeated values create ambiguity. Inspect matching blocks and decide according to the meaning of your elements rather than treating ratio() or a default setting as universal.
The autojunk constructor parameter is documented for SequenceMatcher in Python 3; it was added in Python 3.2. See the difflib documentation.
AI assistance disclosure: This article was drafted with AI assistance and checked using the synthetic example shown below.
from difflib import SequenceMatcher
left = ["A"] * 120 + ["B"] * 120
right = ["B"] * 120 + ["A"] * 120
def block_rows(autojunk):
matcher = SequenceMatcher(None, left, right, autojunk=autojunk)
return [(block.a, block.b, block.size) for block in matcher.get_matching_blocks()]
def format_rows(rows):
return ", ".join(f"Match(a={a}, b={b}, size={size})" for a, b, size in rows)
with_autojunk = block_rows(True)
without_autojunk = block_rows(False)
assert with_autojunk == [(240, 240, 0)]
assert without_autojunk == [(0, 120, 120), (240, 240, 0)]
print(f"autojunk=True:{format_rows(with_autojunk)}")
print(f"autojunk=False:{format_rows(without_autojunk)}")autojunk=True:Match(a=240, b=240, size=0)
autojunk=False:Match(a=0, b=120, size=120), Match(a=240, b=240, size=0)