Confusing CNNs on NPUs with LLMs using FI; or, how much AI can we fit?

[ Abstract ]

Voltage-glitching a commercial NPU to make an edge-AI camera see phantoms, or nothing at all, with LLM agents left to search the fault space overnight. Why neural accelerators are an emerging hardware attack surface.

Edge AI systems power everything from smart cameras and drones to robots and autonomous machines. These devices rely on neural processing units (NPUs) to run computer vision models (CNNs) that decide what the system "sees." But what if you could convince the hardware to see something else… or nothing at all? And what if you can have an LLM do the heavy lifting? Is there no limit to AI relevant abbreviations in a talk title?

In this talk, we show how classic hardware fault injection techniques can manipulate AI inference on real edge devices. By injecting precisely timed voltage glitches on a commercial NPU SoC, we can perturb neural network execution without modifying the model, firmware, or software stack.

Using a Rockchip RK3568 platform running YOLOv5, we demonstrate a range of effects: objects that suddenly disappear, phantom detections appearing out of nowhere, drifting bounding boxes, and the occasional confidently wrong answer. The system keeps running happily, completely unaware that its perception has been tampered with.

Exploring the enormous fault parameter space, however, is tedious for humans. So instead of turning knobs manually, we let LLM-driven agents take over. These agents analyze detection outputs, adapt glitch parameters, and autonomously search for effective fault conditions, running overnight while we sit back (or sleep).

The talk walks through the full attack workflow: observing inference execution through electromagnetic emanations, aligning glitches with neural network stages, and using autonomous agents to optimize the attack. We conclude with implications for the security of edge AI systems and why neural accelerators may represent an emerging hardware attack surface.