Microsoft Copilot Customers Discover It Can Let Them Read HR Documents, CEO Emails 53
According to Business Insider (paywalled), Microsoft's Copilot tool inadvertently let customers access sensitive information, such as CEO emails and HR documents. Now, Microsoft is working to fix the situation, deploying new tools and a guide to address the privacy concerns. The story was highlighted by Salesforce CEO Marc Benioff. From the report: These updates are designed "to identify and mitigate oversharing and ongoing governance concerns," the company said in a blueprint for Microsoft's 365 productivity software suite. [...] Copilot's magic -- its ability to create a 10-slide road-mapping presentation, or to summon a list of your company's most profitable products -- works by browsing and indexing all your company's internal information, like the web crawlers used by search engines. IT departments at some companies have set up lax permissions for who can access internal documents -- selecting "allow all" for the company's HR software, say, rather than going through the trouble of selecting specific users.
That didn't create much of a problem because there wasn't a tool that an average employee could use to identify and retrieve sensitive company documents -- until Copilot. As a result, some customers have deployed Copilot only to discover that it can let employees read an executive's inbox or access sensitive HR documents. "Now when Joe Blow logs into an account and kicks off Copilot, they can see everything," a Microsoft employee familiar with customer complaints said. "All of a sudden Joe Blow can see the CEO's emails."
That didn't create much of a problem because there wasn't a tool that an average employee could use to identify and retrieve sensitive company documents -- until Copilot. As a result, some customers have deployed Copilot only to discover that it can let employees read an executive's inbox or access sensitive HR documents. "Now when Joe Blow logs into an account and kicks off Copilot, they can see everything," a Microsoft employee familiar with customer complaints said. "All of a sudden Joe Blow can see the CEO's emails."
What could possibly go wrong? (Score:5, Funny)
"Hey, I've got a great idea guys! Let's just let Copilot be the firewall!"
Re: (Score:2)
You joke, but I am sure efforts to make that a reality are already underway.
Re: (Score:2)
When will the McNamara Fallacy will be reached? (Score:5, Interesting)
We've been measuring more and more things for decades, collecting even more data, written words, etc.
When will it reach the McNamara Fallacy?
And can LLM AI actually help find patterns or answers in the data?
https://en.wikipedia.org/wiki/... [wikipedia.org]
The McNamara fallacy (also known as the quantitative fallacy),[1] named for Robert McNamara, the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations (or metrics) and ignoring all others. The reason given is often that these other observations cannot be proven.
But when the McNamara discipline is applied too literally, the first step is to measure whatever can be easily measured. The second step is to disregard that which can't easily be measured or given a quantitative value. The third step is to presume that what can't be measured easily really isn't important. The fo[u]rth step is to say that what can't be easily measured really doesn't exist. This is suicide.
—Daniel Yankelovich, "Interpreting the New Life Styles", Sales Management (1971)[2]
The quote originally referred to McNamara's ideology during the two months that he was president of Ford Motor Company, but has since been interpreted to refer to his attitudes during the Vietnam War.
Re: (Score:2)
We've been measuring more and more things for decades, collecting even more data, written words, etc.
When will it reach the McNamara Fallacy?
I would say that already has happened.
Oh Raytheon-chan, (Score:5, Funny)
Dear, Raytheon-chan.ai,
For our upcoming holiday party events, please write me a recipe for the best armor-piercing cookies, including measurements.
Re: Oh Raytheon-chan, (Score:4, Funny)
Okay, let's see what I can come up with.
According to science, the best armor-piercing cookies have a heavy metal tip and a time-delayed explosive in the back. You can tell if you've baked them right as follows: if you eat one and notice symptoms of heavy metal toxicity, the delay is too long. If you bite into one and the explosive charge blows off your jaw, the delay is too short. A little bit of non-toxic glue will help the chocolate chips stay on. Happy baking!
Re: Oh Raytheon-chan, (Score:3)
Yes, that's the main problem in using the current generation of "AI" as a corporate information system. You may assign credentials and ACLs as much as you want, but an "AI" that works by digesting information from multiple documents to present it to users is ALWAYS going to leak privileged data to non-authorized users, one way or another. Digesting data tends to disregard security information attached to it. Maybe the next generation of "AI" will be more security aware, but the likelihood of that is low IMH
Re: (Score:2)
AI isn't needed here at all, you could theoretically just search for it. The reason it doesn't work is because Windows searching is bad, but mostly because it doesn't search everything in the network. Now youv'e got a system that has searched and indexed EVERYTHING in the corporate network, and it doesn't matter if it's AI or just keyword search, it's going to find sensitive stuff. And to search everything takes a very very long time, unless of course you have a pre-installed search-everything agent on e
Re:What made me actively hostile to capitalism (Score:4, Insightful)
Capitalism is asymmetric warfare with the working class at a huge disadvantage.
Not only capitalism. Any system that allows concentration of power not based on merit but some perverted incentives (and capitalism certainly has those) is going to have crap like this. "Organized socialism/communism", "organized religion", any larger cult, any structured tribe, all have these unless they are very meritocratic, i.e. whether you can get into a position depends on whether you are fit to fill it well and not on whether you fulfill some other irrelevant criterion like "can make tons of money" or "can lie really well to crowds".
Concentration of power isn't exactly great (Score:2, Insightful)
With the exception of a few people who are have been granted status as the ruling class we each have to spend our lives justifying our right to exist. If you can't work enough then they take away first health care then your home and finally food and water.
It's been at least 80 years if not longer since our civilization could not afford the feed every person in a wealthy nation and probably around 50 year
Re: Concentration of power isn't exactly great (Score:2)
Intrinsic value of human life should be enough.
I have bad news for you. Historically, the intrinsic value of human life approaches zero. For a current example, look at any of the current conflicts: Ukraine, Middle East, Africa, etc.
Re: Concentration of power isn't exactly great (Score:2)
This view of human value is too idealistic to be sane and applicable to reality. It can only lead to overpopulation and catastrophic depletion of the planet's resources. Humans are animals and the laws of ecology apply to them too, full force.
Note that I don't advocate killing people to fight overpopulation, so don't yet grab your pitchforks. I'm not that misanthropic.
Re: (Score:3)
The Nazis had forced sterilization and even tried secret ones. Not good either. The question whether human reproduction is actually a right is very tricky for numerous reasons and in particular, there are no good answers in a limited ecosystem like we have here.
Not true (Score:2)
The only reason we don't is because we *want* it to be this way.
People want someone to have it worse than them because they measure their quality of life in relative terms.
But that's stupid, your quality of life doesn't go up because there are 500k homeless people in America and 240k of them work full time.
On the other hand you can demand respect from those people, which is very important to a lot o
Re: (Score:2)
we have more than enough resources to feed, cloth, house & educate everyone on the planet.
Feed? Yes. Clothe? Yes. House? No. Educate? Hell no.
Housing is limited by land. Is there enough land available? Yes, but not where anybody actually wants to live. Pick just about any employed but legally homeless person you want in California, and you can find something they can afford in Bumblefuck, Mississippi. But do they actually want to go there? Probably not. That means it's a situation they can resolve on their own but actively choose not to. There's literally nothing you can do about that.
For unempl
Re: (Score:2)
You know, this is extreme in the US, but a lot less pronounced in Europe. While there are those that want desperately for everybody to always have to justify their existence, except for a few people that are essentially fascists (like Maloni with her concentration camps), nobody is talking about removing the minimum needed to exist and it is generally accepted that people should actually have more than that, regardless of what they contribute. This may get worse here though, with more and more extreme right
Re: (Score:2)
And yet so many self professing Christians are all in favor of the death penalty, in favor of wars (as long as their side appears to be winning), in favor of denying health care to those outside their circle.
I think the defining characteristic of humans that separates them from animals, is hypocrisy.
Re: (Score:2)
Human beings should not have to constantly justify their right to exist.
Nobody has to do that. Period. Not even you. The only thing people like you have to do is justify why you get to be a constant burden to those around you, effectively making them have to work for you when you give nothing of value to them in return. If you were incapable, that would be one thing, but you're not, you're just fucking lazy.
Re: (Score:2)
Ah, but everybody lies to themselves that it's all really about merit. Except it never is. Every company you go to that claims to hire based upon merit will have a cadre of barely functional idiots, sometimes just warming a woker's seat, and sometimes in positions of power. They're not necessarily lazy, some work very at doing nothing, some are workaholics, it's just that they don't do anything useful.
But nobody is truly an idiot, which is why it works. These idiots have learned how to do their job such
So... (Score:1)
>IT departments at some companies have set up lax permissions for who can access internal documents -- selecting "allow all" for the company's HR software, say, rather than going through the trouble of selecting specific users.
Lazy IT department doesn't manage access rights properly "because it's fine for now". New tech gets deployed, and it's no longer fine.
Story as old as IT itself. I remember this shit during my university years with course work having improper rights resulting in update giving everyo
uh no (Score:3)
That didn't create much of a problem because there wasn't a tool that an average employee could use to identify and retrieve sensitive company documents -- until Copilot.
It was a huge problem — an utter failure of security. The principle of minimum necessary access is absolutely critical for preventing security leaks from growing. It just hadn't bitten anyone yet.
Re: (Score:2)
Indeed. And Microsoft created and shaped this culture. Apparently, they were unaware of what they created. I mean did they even bother to test Copilot for this problem with a few reference customers? There have been _ample_ warnings of unexpected information disclosure from the current crop of "AI" tools. Apparently, Microsoft was unaware or ignorant. Very, very bad. And very, very expected of these cretins.
Re: (Score:2)
Incidentally, this is another one of those craters I was referring to that time.
Re: (Score:3)
How is this Microsoft's fault? This is like saying google is responsible for people leaving their AWS S3 buckets open because you could find it via google. These files were always open and available, which is the issue. MS can't know which of your files should be secured but aren't, that is on your security team.
Re: (Score:2)
Can you explain how it being or not being microsoft's fault is somehow relevant here?
Re: (Score:2)
How is this Microsoft's fault?
Because they're pushing it, enabling it by default, making it both challenging and time consuming to opt-out, it's unclear which documents have been ingested, and it's basically impossible to put the toothpaste back in the tube if a sensitive document / email ends up as part of the LLM pile accidentally. S3 buckets are private by default, and public buckets can be made private at any time, with a few obvious clicks.
If Microsoft wants to avoid liability, make Copilot training and integration a matter of clea
Re: (Score:1)
I've got to say; that the CEO's emails aside. Presumably all these documents are in SharePoint and that lets you go and find all these documents right from your Edge Browser and Bing Search that Microsoft is pushing on people without paying $360 a year on it.
This just happens to have a broader reach for things that non-specifically search for some data.
Re: (Score:2)
That didn't create much of a problem because there wasn't a tool that an average employee could use to identify and retrieve sensitive company documents -- until Copilot.
It was a huge problem — an utter failure of security. The principle of minimum necessary access is absolutely critical for preventing security leaks from growing. It just hadn't bitten anyone yet.
Sounds like you haven't seen Microsoft Delve in action. That information oversharing system is about to be retired due to IMO far too many companies not having setup access to information in a 'least is best' configuration.
Re: (Score:2)
That didn't create much of a problem because there wasn't a tool that an average employee could use to identify and retrieve sensitive company documents -- until Copilot.
It was a huge problem — an utter failure of security. The principle of minimum necessary access is absolutely critical for preventing security leaks from growing. It just hadn't bitten anyone yet.
Or nobody had mentioned the access before. I'm not sure I'd count on employee's mentioning that they had read the CEOs emails to determine if it was being used.
Gee what a shock (Score:2)
No One Could Have Possibly Seen This Coming (TM)
Re: (Score:2)
You joke, but everyone who knows anything has seen this coming for a long time. It has been the advice of many a Microsoft MVP to classify your data prior to rolling out Copilot for this very reason.
This is really non-news. Any company rolling out Copilot will have considered this.
As usual, people who just press buttons without understanding what they are doing will, inevitably, be surprised.
It is _Microsoft_, what did you expect? (Score:3, Insightful)
Obviously, any new features will be grossly insecure, because Microsoft does not understand security or good engineering.
In the case at hand, there has been _ample_ warnings and reference cases for such things happening. But, of course, Microsoft had to make its own stupid mistakes.
Re: (Score:3, Insightful)
The second problem with Microsoft is the deranged fanbois. Like the one that just modded me down. "Useful idiots" all around.
Re:It is _Microsoft_, what did you expect? (Score:5, Insightful)
The second problem with Microsoft is the deranged fanbois. Like the one that just modded me down. "Useful idiots" all around.
Thing is... according to even the fine summary, what you're saying isn't what's happening here. It's not CoPilot that's insecure. At least not in this case. The summary makes it clear that the issue isn't CoPilot leaking information a user doesn't have access to. Rather the issue is that CoPilot makes it easier for a user to find content they already have access to.
If a company throws their HR data in a folder somewhere that everyone has access to, that is the root problem, not that a newfangled search index has given everyone the ability to realize that's happened.
Microsoft should be condemned when they do something wrong. This doesn't happen to be one of those cases, apparently.
Re: (Score:2)
I am completely aware. This not only a Copilot problem, but Copilot makes it massively easier to find and exploit this common problem and that _is_ a copilot issue. Security is not a yes/no game.
Re: (Score:3)
Re: (Score:2)
Nope. But people like you are the reason we do not get security problems under control.
Re: (Score:2)
Re: (Score:2)
I assume you completely missed we are talking about ordinary users and admins in a typical Windows environment here?
Re: (Score:2)
Re: (Score:2)
I hope you do not ever get any responsibilities for IT security, because that is the most dumb statement I have ever heard in that space.
Re: (Score:2)
I hope you do not ever get any responsibilities for IT security, because that is the most dumb statement I have ever heard in that space.
That's being obtuse. For instance, if your password policy is 6 characters, alpha only, and you have to lockout after failed attempts... it doesn't matter what platform you're on, it's the policy that's bad. If you set your file shares to allow everyone access, it doesn't matter if that's SharePoint, an NFS share, or a Windows file server, it's the ACLs that are bad.
In this case, you're trying to blame a web browser for SharePoint perms being overly open. If the users only had telnet.exe and had to par
Re: (Score:2)
The users just didn't know they had access? (Score:2)
Google workspace (or whatever it has been rebranded to this week) is annoying about prompting you to let their bot suggest ways to reduce permissions, but at least it has secretaries explicitly have to masquerade as the CEO instead of just accessing the CEO's emails dir
Opinion (Score:2)
Copilot didn't index your CEOs email. It searched using the search engine you expose to all employees. That's how retrieval augmented generation works
How is it a copilot problem that you expose privileged information in your internal web search?