Cybercriminals can use inaudible background sounds in audio and video files to hack smart speakers and AI assistants and access personal information, a new study warns.
Modern voice assistants are powered by AI tools called large language models which come with audio and text tightly integrated.
Research shows that cleverly crafted prompts called “jailbreaks” can bypass built-in safety guidelines and ethical restrictions of AI assistants.
Hackers are known to use jailbreaks to make AI chatbots fulfil requests they are programmed to refuse, such as generating hate speech, assisting with cyberattacks, and revealing restricted information.
While text prompts are widely studied, the security risks of audio jailbreaks and their manipulative effects on AI systems remain underexamined, a team of cybersecurity researchers from China and Singapore say.
Such “adversarial audio”, undetectable to the human ear, can trick AI models into performing tasks they aren’t supposed to otherwise. “In this work, we reveal a previously overlooked threat, auditory prompt injection,” the researchers note in a yet-to-be peer-reviewed study posted on arXiv.
Hackers using audio jailbreaks can covertly provide limited input to hijack an AI model’s behaviour.
Although this kind of attack is more constrained than a text jailbreak, the researchers say, it can be “potentially more harmful”.
The researchers developed a method to use imperceptible audio to hijack audio-based AI models like smart speakers.
They tested this method, called Audiohijack, on 13 state-of-the-art audio-based AI models and found that a majority could be covertly and successfully hijacked regardless of what the user’s prompts said.
“The attack induces misbehaviours ranging from simple prompt refusal to complex tool misuse, achieving average success rates of 79 to 90 per cent,” the researchers say in the study.
The “adversarial audio” could manipulate the AI agents into executing unauthorised actions, including downloading malicious files and revealing user information via email.
“No dedicated defences exist for this new threat,” the researchers warn as on-device integration of AI becomes common, widely deployed on electronics equipment like mobile phones and smart speakers.
The findings, the researchers say, reveal fundamental vulnerabilities in the audio-text integration of AI models.
“In these settings, auditory prompt injection could interact with system components and third-party apps to enable broader compromise,” they say.
“Future work should extend the evaluation to system-level applications and real devices to assess the practical risk better.”
