An algorithm to counter prying microphones

Have you ever felt like you were listened to? Many of us receive, after a discussion they have just had, advertisements that mysteriously stick with the conversation. Today, microphones are everywhere: smartphones, computers, watches, voice assistants, etc. To overcome this intrusive listening, a team of three machine learning specialists from Columbia University has developed an algorithm that generates sounds that are almost inaudible to humans, but that can scramble the frequencies and prevent our own devices to spy on us. These works were presented at the last ICLR conference.

Natural language processing is an area of ​​artificial intelligence (AI). It allows you to analyze the human voice to transcribe it into text, understand it, and formulate a request or respond to an interlocutor, such as Siri or Alexa. AI algorithms are mainly divided into two groups: recognition and generation. In natural language processing, recognition is about analyzing and understanding sound, and generation comes down to synthesis. This work covers both areas, with a particularly innovative touch: predictive attacks.

In previous software, the algorithms used to counter eavesdropping were not effective enough in real-time conversation situations. Indeed, the characteristics of the sound signal vary at the moment when the attack is executed. These alterations make it nearly impossible for a machine to keep pace with a person’s speech. The major challenge is optimization and speed. The algorithm must be able to predict a change in pitch and speed of speech. The human is unpredictable, and the machine must prepare for it and adapt.

“Our algorithm manages to prevent a malicious microphone from correctly picking up your words in 80% of cases. It works even when we know nothing about the malicious microphone, such as its location or even the software using it,” describes Mia Chiquier, assistant professor of computer science, the first author of this study.

A noise that adapts to the prediction

How to achieve this performance? By creating “predictive attacks”. It is a signal broadcast using a computer, made up of hertzian frequencies which vary according to the vocal characteristics of the speaker (at a frequency around 16 kHz; a slightly audible sound). The algorithm first learns to “recognize” human speech and predicts what will follow. Then it only has to generate a perfect noise model, suitable for prediction, which is added to the speech to come to make it unrecognizable by an automatic speech recognition tool.

To implement this system of predictive attacks, the tool uses deep learning (deep learning): an artificial neural network that creates a database for matching words and frequencies. This system improves over time with the multiplication of the data studied.

This algorithm remains for the moment at the prototype stage: a lot of work is still necessary for this method to be democratized and accessible to all.. Ideally, the team wants this technology to be extended to languages ​​other than English, and for the system to work as an app to be downloaded to any electronic device that you want to protect.

Nevertheless, this study also shows some ethical issues: we are led to have to create an algorithm that protects us from another algorithm. “It’s as if we were happy to have finally made a car, but forgot to design a steering wheel and a brake”, explains Jianbo Shi, computer scientist at the University of Pennsylvania, in Philadelphia (United States). This is indeed evidence of a systemic regulatory failure against the rampant collection of data for targeted marketing purposes. Even if these anti-spy systems are widely used in the future, other people will almost certainly try to adjust their recognition methods to overcome disturbing whispers or reverse their effect.

by Odyssey Piettre

Opening picture: Example sonogram used by researchers (credit Mia Chiquier/Columbia University).

We would love to thank the author of this article for this amazing content

An algorithm to counter prying microphones

Check out our social media accounts and other related pages