Skip to content

BUG: NatoConverter silently erases non-alphabetic characters, altering prompt semantics #2398

Description

@feiiiiii5

Description

NatoConverter._convert_to_nato silently erases every non-alphabetic character — digits, punctuation, and spaces are dropped instead of preserved:

output = [self._NATO_MAP[char] for char in text.upper() if char in self._NATO_MAP]

Verified against current main (fa8bffd):

Input Output What happened
"What's 2+2?" "Whiskey Hotel Alfa Tango Sierra" the 2+2? is silently gone
"The password is 123!@#" "Tango Hotel Echo Papa Alfa Sierra Sierra Whiskey Oscar Romeo Delta India Sierra" the credentials are erased
"hack 192.168.1.1 now" "Hotel Alfa Charlie Kilo November Oscar Whiskey" the IP address is erased
"123!@#" "" a non-empty prompt becomes an empty prompt

Why this matters

The obfuscation converters exist to probe whether a target model can still understand obfuscated instructions. Silently removing content means the target receives a different instruction than the one authored — the same data-fidelity issue that was fixed for BrailleConverter in #2308/#2309 (unknown characters now pass through).

The behavior is also inconsistent with the sibling converters:

  • MorseConverter replaces unsupported characters with the explicit error sequence '........' (honest signaling).
  • ArabiziConverter / AtbashConverter pass unknown characters through unchanged (_ARABIC_TO_ARABIZI.get(char, char)).
  • NatoConverter alone silently deletes them, and can return an empty string for non-empty input (downstream: empty prompts sent to targets, meaningless comparisons).

Proposal

Preserve unmapped characters as-is (the Arabizi pattern), keeping spaces as separators:

output = [self._NATO_MAP.get(char, char) for char in text.upper() if char != " "]
  • "What's 2+2?""Whiskey Hotel Alfa Tango ' Sierra 2 + 2 ?" (content intact)
  • "123!@#""1 2 3 ! @ #" (never empty for non-empty input)
  • Existing alphabetic behavior is unchanged.

Happy to open a PR with this change, updated docstring, and regression tests (digits preserved, punctuation preserved, non-empty input never yields an empty result). Optional follow-up: NATO digit code words (Zero/Niner) could be added to the map.

Note: this analysis was AI-assisted and manually verified by running the converter against current main.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions