Signal President Calls Out Agentic AI As Having 'Profound' Security and Privacy Issues (techcrunch.com) 8
Signal President Meredith Whittaker warned at SXSW that agentic AI poses significant privacy and security risks, as these AI agents require extensive access to users' personal data, likely processing it unencrypted in the cloud. TechCrunch reports: "So we can just put our brain in a jar because the thing is doing that and we don't have to touch it, right?," Whittaker mused. Then she explained the type of access the AI agent would need to perform these tasks, including access to our web browser and a way to drive it as well as access to our credit card information to pay for tickets, our calendar, and messaging app to send the text to your friends. "It would need to be able to drive that [process] across our entire system with something that looks like root permission, accessing every single one of those databases -- probably in the clear, because there's no model to do that encrypted," Whittaker warned.
"And if we're talking about a sufficiently powerful ... AI model that's powering that, there's no way that's happening on device," she continued. "That's almost certainly being sent to a cloud server where it's being processed and sent back. So there's a profound issue with security and privacy that is haunting this hype around agents, and that is ultimately threatening to break the blood-brain barrier between the application layer and the OS layer by conjoining all of these separate services [and] muddying their data," Whittaker concluded.
If a messaging app like Signal were to integrate with AI agents, it would undermine the privacy of your messages, she said. The agent has to access the app to text your friends and also pull data back to summarize those texts. Her comments followed remarks she made earlier during the panel on how the AI industry had been built on a surveillance model with mass data collection. She said that the "bigger is better AI paradigm" -- meaning the more data, the better -- had potential consequences that she didn't think were good. With agentic AI, Whittaker warned we'd further undermine privacy and security in the name of a "magic genie bot that's going to take care of the exigencies of life," she concluded. You can watch the full speech on YouTube.
"And if we're talking about a sufficiently powerful ... AI model that's powering that, there's no way that's happening on device," she continued. "That's almost certainly being sent to a cloud server where it's being processed and sent back. So there's a profound issue with security and privacy that is haunting this hype around agents, and that is ultimately threatening to break the blood-brain barrier between the application layer and the OS layer by conjoining all of these separate services [and] muddying their data," Whittaker concluded.
If a messaging app like Signal were to integrate with AI agents, it would undermine the privacy of your messages, she said. The agent has to access the app to text your friends and also pull data back to summarize those texts. Her comments followed remarks she made earlier during the panel on how the AI industry had been built on a surveillance model with mass data collection. She said that the "bigger is better AI paradigm" -- meaning the more data, the better -- had potential consequences that she didn't think were good. With agentic AI, Whittaker warned we'd further undermine privacy and security in the name of a "magic genie bot that's going to take care of the exigencies of life," she concluded. You can watch the full speech on YouTube.
Haha, will get used anyways (Score:3)
I don't want to mention any names, but some of them read this website .
Re: (Score:2)
And when their bank accounts turn up empty, they will of course blame somebody else for their stupidity.
Some sort of hardware firewall may be necessary (Score:2)
Hopefully, the developers of AI agents will build in security, but it may be unreliable, especially at first
A separate computer with no access to sensitive data and a hardware based firewall might be a good idea in the early days
Controlled Agency (Score:1)
Feature, not a bug (Score:2)
"And if we're talking about a sufficiently powerful ... AI model that's powering that, there's no way that's happening on device," she continued. "That's almost certainly being sent to a cloud server where it's being processed and sent back. So there's a profound issue with security and privacy that is haunting this hype around agents
"Cool!" said every single venture capitalist within earshot. Followed by "I'm all ears. Tell me more. Want some money? Write me a proposal!"
As a total aside, I'm happy to be watching the video. I'd forgotten what a gem Whittaker is.
Agents are a ploy to get your most personal data (Score:2)
Re: Agents are a ploy to get your most personal da (Score:2)
What fool would run that on their main OS? (Score:2)
Why would anyone with even modest computer literacy not use a throwaway OS install or virtual machine or boot from an external hard drive that cannot access the main OS for this?
Who registers personal software to be used on a main OS under their real name? That was stupid thirty years ago and hasn't gotten wiser.
AI agents can electronically REQUEST user data but that in no way means you need to expose that data to the OS install they run on. Anyone on Slashdot should damn well have more than one PC and know