The Terrible Truth About Alexa
This week, I read through a history of everything I’ve said to Alexa, and it felt a little like reading an old diary. Until I remembered that the things I’ve told Alexa in private are stored on an Amazon server and possibly read by an Amazon employee. This is all to make Alexa better, the company says. However, many see this as surveillance. Alexa, they argue, is a spy hiding in a wiretapping device.
The debate over whether or not Alexa or any voice assistant is spying on us is years old and isn’t going away. Privacy advocates have filed complaints with the Federal Trade Commission (FTC), alleging these devices violate the Federal Wiretap Act. Journalists have investigated the dangers of always-on microphones and AI voice assistants. Skeptical tech bloggers like me have argued these things are overloaded with privacy violations. Recent news about Amazon employees reviewing certain Alexa commands suggests the situation is worse than we thought.
It’s starting to feel like Alexa and other voice assistants are destined to spy on us because that’s how they were designed. These systems rely on machine learning and artificial intelligence to improve over time. The technology is prone to error, and the companies behind them are constantly thinking of new ways to exploit users for profit. The government struggles to understand what’s going on, making regulation seem impossible.
The situation isn’t completely dire. This technology could be really cool if we paid closer attention to what’s happening. But it’s pretty complicated.
Never-ending Errors
One fundamental problem with Alexa or other voice assistants is that the technology is prone to fail. Devices like the Echo come equipped with always-on microphones that are only supposed to record when you want them to listen. Many are designed to start recording after you’ve said the wake word. Anyone who has used Alexa knows it doesn’t always work like this. Sometimes the software hears random noise, thinks it’s the wake word, and starts recording.
The extent to which false positives are a problem became evident when I started reading through my history of Alexa commands on Amazon’s website. Most of the entries are dull: “Hey Alexa;” “Show me an omelet recipe;” “What’s up?” But among the mundane were messages that said, “Text not available—audio was not intended for Alexa.” These are things Alexa heard that it should not have heard. In other words, they’re errors.
Voice assistants picking up stray audio is an inevitable defect in the technology. The sophisticated computer program that can understand anything you say is behind a simple one trained to hear a wake word and then send commands to the smarter computer. The problem is that the simple computer often doesn’t work right, and people don’t always know there’s a recording device in the room. That’s how we get Echo-based nightmares like the Oregon couple who inadvertently sent a recording of an entire conversation to an acquaintance. Amazon has been working on improvements, but it’s hard to imagine the system will ever be flawless.
“There is a microphone in your house, and you do not have final control over when it gets activated,” Dr. Jeremy Gillula of the Electronic Frontier Foundation (EFF) told me. “From my perspective, that’s problematic from a privacy point of view.”
This sort of thing happening is bad luck, although it’s more common than most people would like. What’s worse than glitches is the behind-the-scenes workflow that reveals users’ interactions with voice assistants to strangers. Bloomberg recently reported that a team of Amazon employees has access to Alexa users’ geographic coordinates to improve the voice assistant’s abilities. This revelation came after Bloomberg reported that thousands of Amazon employees analyze users’ Alexa commands to train the software. They can overhear compromising situations and sometimes make fun of what people say.
Amazon pushed back hard against these reports. A spokesperson said Amazon only annotates “an extremely small number of interactions from a random set of customers to improve the customer experience.” These recordings are kept in a protected system that uses multi-factor authentication so that “a limited number” of carefully monitored employees can access them. Bloomberg suggests the team numbers in the thousands.
But for Alexa and other voice assistants to work, some human review is necessary. This training could prevent future errors and lead to better features. Amazon isn’t the only company using humans to review voice commands. Google and Apple also employ teams to review what users say to their voice assistants. Sure, the human element is creepy, but it’s also essential for developing these technologies.
“For really hard cases, you need a human to tell you what was going on,” Dr. Alex Rudnicky, a computer scientist at Carnegie Mellon University, said in an interview. Rudnicky has been developing speech recognition software since the 1980s and has led teams in the Alexa Prize, an Amazon-sponsored contest for conversational AI. He believes it’s incredibly unlikely for a voice command to get traced back to one individual.
“Once you’re one out of 10 million,” Rudnicky said, “it’s hard to argue that someone’s going to find it and trace it back to you and find out stuff about you that you don’t want them to know.”
This idea doesn’t make the idea of a stranger reading your daily thoughts or knowing your location history feel any less creepy. It might be uncommon for a voice assistant to record me accidentally, but the systems don’t seem smart enough to wake up with 100 percent accuracy. The fact that Amazon catalogs and makes available all the Alexa recordings it captures—accidental or otherwise—makes me feel terrible.
The Privacy Problem Nobody Wants to Fix
In recent conversations, technology and privacy experts told me we need stronger privacy laws to tackle some of these problems with Alexa. The amount of your personal data an Echo collects is bound by terms that Amazon sets, and the United States lacks strong federal privacy legislation like Europe’s General Data Protection Regulation (GDPR). In other words, the companies building voice assistants are more or less making the rules.
So I find myself circling back to a few questions. Who’s looking after the users? Why can’t I opt in to letting Amazon record my commands instead of wading through privacy settings looking for ways to stop sending my data to Amazon? And why are my options for opting out limited?
In the Alexa privacy settings, you can opt out of letting Amazon use your recordings to develop new features and improve transcriptions. You cannot opt out of allowing Amazon to retain your recordings for other purposes.
Amazon could use voice recognition to identify you and build extensive profiles about who you are, what you do, and your habits. He suggests Amazon could make a business out of this, knowing who you are and what you like by the mere sound of your voice. Unlike facial recognition, voice recognition could work without ever seeing you. It could work over phone lines. In a future where internet-connected microphones are present in more rooms, a system like this could always be listening. Several researchers I talked to brought up this dystopian idea and lamented its imminent arrival.
Such a system is hypothetical, but all the pieces are in place. There are tens of millions of devices with always-on microphones in homes and public places. They’re allowed to listen and record at certain times. These AI machines are prone to errors and will only get better by listening more, sometimes letting humans correct their behavior. Without government oversight, who knows how the system will evolve.
We wanted a brighter future than this. Talking to your computer seemed cool in the 90s and was a significant part of the Jetsons’ lifestyle. But it seems like an unavoidable truth that Alexa and other voice assistants are bound to spy on us, whether we like it or not. The technology is designed in such a way that it can’t be avoided, and without oversight, it will probably get worse.
Maybe it’s foolish to think Amazon and other companies are worried about privacy. Maybe they’re working on fixing problems caused by error-prone tech and addressing the anxiety people feel when they see devices like the Echo recording them, sometimes without realizing it. Maybe Congress is working on laws to hold these companies accountable.
Inevitably, the future of voice-powered computers doesn’t have to be dystopian. Talking to our gadgets could profoundly change how we interact with technology if everyone were on board with how it was being done. Right now, that doesn’t seem to be the case. Ironically, the fewer people we have helping develop tech like Alexa, the worse Alexa will be.
