Security Through Obscurity Is Dead and LLMs Killed It

[ Abstract ]

Two case studies in LLM agents ripping through obscurity: reverse-engineering the currency detectors in photocopiers and image editors to build an unphotocopiable cat, and auto-building a faithful QEMU emulator for a laser printer. His conclusion: obscurity now buys days, not years.

Seasoned hackers have always used "security through obscurity is not security" as a mantra to discourage building systems whose security depends on no one caring enough to look too closely at the details. But in practice these systems abound, and they often remain (publicly) unbroken because human expert time is scarce. In this talk we'll show how LLMs disrupt this equilibrium via two case studies: 1) Reverse engineering currency detection algorithms used by photocopiers and image editing tools, and then attacking them to produce an image of my cat that cannot be edited or photocopied; and 2) Automatically building a highly faithful QEMU emulator for an HP M551dn color laser printer firmware, to the point where you can put it on a network, print to it, and get back faded PNGs if it's running out of emulated toner. We conclude that the security afforded by obscure environments and algorithms can now only provide a few days of protection from anyone with curiosity and tokens to burn, and discuss the implications for anyone building robust systems.