The Paper That Changed Everything
Looking back on May 2022, at the IEEE Symposium on Security and Privacy in San Francisco, four researchers Nicholas Boucher, Ilia Shumailov, Ross Anderson, and Nicolas Papernot presented a paper that, years later, we can now see fundamentally reshaped how we think about NLP security. Their work, "Bad Characters: Imperceptible NLP Attacks," was a prophetic work that birthed an entire sub-industry.
The authors asked what if the gap between what humans see and what machines process could be weaponized?
Their answer exposed a vulnerability class so fundamental that it affected virtually every text-based NLP system deployed at scale. Google Translate, Microsoft Azure ML, IBM's classifiers, Facebook's models all vulnerable. The paper's title itself contained 1,000 invisible characters, imperceptible to human readers but devastating to machine processing... super meta!
Looking back, the taxonomy the authors created has proven to be more then just enduring:
The experiment worked with five models, three commercial platforms, five NLP tasks all broken with just three invisible character injections. Every tested system was vulnerable.
Boucher brought implementation genius. Shumailov understood how to break models architecturally. Anderson a founding father of security engineering connected this to decades of input sanitization failures. Papernot grounded it in adversarial ML literature. Lightning in a bottle.
The research questions it opened continue driving PhDs today:
- Perfect defenses without performance degradation? (Still unsolved)
- Architectural immunity in transformers? (Active research)
- Cross-lingual vulnerabilities? (Devastatingly effective)
- Supply chain poisoning detection? (The nightmare scenario)
But more remarkably, it validated an entire commercial ecosystem: scanner tools, MLaaS security layers, NLP security audits, compliance frameworks, training programs. A single paper that created tangible market demand measured in millions.
Most security papers age poorly. "Bad Characters: Imperceptible NLP Attacks" exposed a fundamental architectural gap which represents the disconnect between visual rendering and byte-level processing. As long as we use Unicode and process byte streams, this vulnerability persists. You can't patch your way out of architectural problems.
Citation count climbing. Unicode Consortium working groups on security profiles. Taught in graduate courses worldwide. Input sanitization becoming standard practice. The paper's influence compounds annually the mark of truly foundational work.
The genius of Boucher, Shumailov, Anderson, and Papernot was recognizing that lessons learned painfully over decades of web security never trust user input, always sanitize, defense in depth had been completely forgotten in the AI revolution's rush to deployment.
Here's to the brilliance of asking the question no one had properly asked before.
Read the full paper: Bad Characters: Imperceptible NLP Attacks
Published: 2022 IEEE Symposium on Security and Privacy (SP)
DOI: 10.1109/SP46214.2022.9833641
Authors: Nicholas Boucher, Ilia Shumailov, Ross Anderson, Nicolas Papernot
This article is part of the Bad Character Scanner blog series exploring the landscape of invisible character vulnerabilities in modern software systems.