Amazon's Q Has 'Severe Hallucinations' and Leaks Confidential Data in Public Preview, Employees Warn (platformer.news) 43
Three days after Amazon announced its AI chatbot Q, some employees are sounding alarms about accuracy and privacy issues. From a report: Q is "experiencing severe hallucinations and leaking confidential data," including the location of AWS data centers, internal discount programs, and unreleased features, according to leaked documents obtained by Platformer. An employee marked the incident as "sev 2," meaning an incident bad enough to warrant paging engineers at night and make them work through the weekend to fix it.
[...] In a statement, Amazon played down the significance of the employee discussions. "Some employees are sharing feedback through internal channels and ticketing systems, which is standard practice at Amazon," a spokesperson said. "No security issue was identified as a result of that feedback. We appreciate all of the feedback we've already received and will continue to tune Q as it transitions from being a product in preview to being generally available."
[...] In a statement, Amazon played down the significance of the employee discussions. "Some employees are sharing feedback through internal channels and ticketing systems, which is standard practice at Amazon," a spokesperson said. "No security issue was identified as a result of that feedback. We appreciate all of the feedback we've already received and will continue to tune Q as it transitions from being a product in preview to being generally available."
guess the current generation of AI's are trending (Score:4, Interesting)
Re:guess the current generation of AI's are trendi (Score:4, Funny)
Re: (Score:3)
So they are not hallucinating, they just tell the Actual Truth (tm) that The Elites (tm) try to keep from us, I get it?
Re: (Score:3)
(holy fuck, what have I done... conspiracy nuttery claiming just exactly this coming in 3, 2, 1...)
Re: (Score:1)
So they are not hallucinating, they just tell the Actual Truth (tm) that The Elites (tm) try to keep from us, I get it?
No, they're buggy shite.
Re: (Score:2)
That is EXACTLY what The Elites (tm) want you to think!
Re: (Score:2)
That would be funny in the ironic way.
Think of the future history students reading about the two Q Anons then..
Re: (Score:1)
If you think half the population is crazy you're likely a brainwashed cult member.
Think about how societies work and what normal means.
Also consider epistemology and humility.
The Dunning-Kruger effect is predictive.
Re: (Score:2)
Re: (Score:2)
If you think half the population is crazy you're likely a brainwashed cult member.
Think about how societies work and what normal means.
There are two very different things that we could talk about when we talk about "crazy". Option one: You insist on believing a bunch of things that are demonstrably false. Option two: You're unable to act within the normal range of behaviour in the society you're living in.
Most of the people in the world believe in some kind of demonstrably false bullshit, usually inspired by religion. Most of the people in the world also act within their society's normal ranges of behaviour.
I'm not sure what conclus
Ooops (Score:2)
Now the other will have to catch up and increase hallucinations as well!
In other news, turns out training a large LLM is not something you can do in a few months...
Re: Ooops (Score:2)
Considering that Q in Star Trek was like a hallucination I'm not surprised.
Re: (Score:1)
Hallucinations aren't such a big deal when your CEO isn't a Science Fiction advocate. Considering this is only accessible through an API, hopefully they can avoid a lot of the media-driven FUD.
Re:Ooops (Score:4, Insightful)
Well, given the "quality" of the average CEO, hallucinations would hardly make their decision-making worse...
Re: (Score:3)
When a CEO hallucinates, we call that him having a "vision", if you don't mind.
How dare you suggest someone raking in a couple millions a year is insane? Our board would look like idiots!
Re: (Score:2)
"Those who have visions should medical aid" (Helmut Schmidt, former German Chancellor), is still the best comment on "visions" I know.
Re: (Score:2)
I was for the longest time thinking I came up with this bonmot, but it was either Schmidt or the Austrian Chancellor Vranitzky who gave that exact reply to someone who said that the SPÖ needs some new "visions".
Either way, it's definitely true.
Re: (Score:2)
Though Vranitzky said it way more ... colloquially: "Wea Visionen hod, dea sulltat si des au'schaun loss'n".
Re: (Score:2)
Indeed. Hehehehehe....
Re: (Score:2)
Is nothing sacred? (Score:2)
Look, I am used to corporations meddling with our political process, but that they now create their own presidential candidate by basically creating an artificial version of the others goes too far!
Re: (Score:2)
Look, I am used to corporations meddling with our political process, but that they now create their own presidential candidate by basically creating an artificial version of the others goes too far!
Jokes on them, many of our politicians were artificially intelligent all along.
Re: (Score:2)
I don't question the artificial, but for the intelligent I want some conclusive proof to believe it.
Re: (Score:2)
I don't question the artificial, but for the intelligent I want some conclusive proof to believe it.
Well, they both hallucinate and turn terribly racist when exposed to social media. Umm, maybe they are just artificial.
Simple advice for the leaks (Score:5, Informative)
Don't put any document in the training that you don't want to see eventually exposed.
The training basically consist in having the network memorize and connect everything you throw at it, codified in a highly compressed form.
Even if you add to the mix instructions telling it to hide some of that information when it's queried from certain angle, you'll never know what other connections have been made that will surface the same information when asked from a different perspective.
Therefore, train any public-facing AI on a need-to-know basis. The most secure information is the one that is not there.
Re: (Score:3)
Yes. But that runs into an issue: These models are trained cheaply and fast, because anything else cannot be financed anymore. Hence they throw in anything they have in data and hope it will not get exposed. That even this approach does not result in a good mode and hallucinations abound (which is a strong indication of too little training data), basically means the whole approach is deeply flawed.
The only way I see to make this approach somewhat useful is training LLMs on very restricted data for one small
Re:Simple advice for the leaks (Score:5, Interesting)
The problem here is the "garbage in - garbage out" problem.
The first generation of AI was trained on human generated content. By definition. Because there wasn't anything but human generated content.
With every generation of AI, more and more content out there is AI generated, and nobody has the time and money to vet and curate that content anymore. Since it's also been shown that it is very difficult to automatically tell AI generated content from human content, until you run into a "WTF is that junk???" moment when you finally get around to seeing some AI content, and the fact that AI can generate far, far faster than even the collective human effort can, more and more content used to train AI will by definition come from AI.
I guess we'll soon run into the problem that AI is learning nearly exclusively from AI. Now, let's assume that 90% of AI generated content isn't total garbage. That means the first generation of AI had 100% sensible content to learn from (i.e. 100% human content). The next generation will have about 91%. The generation after hat, about 81%. and so on.
At what point will AI be reduced to learning more garbage than something that could actually be useful?
It's like going to school in a deep south county where they insist in teaching creationist bullshit. At what point is your education simply and plainly not lining up anymore with reality?
Re: (Score:2)
Yep. This may be a first case of observable "model collapse" observable in the wild. (AI getting drastically more dumb when trained on AI output.) I would have expected this to take a while longer to happen, but that was just a guess and "severe hallucinations" does sound like a very strong indicator for model collapse.
On the plus-side, this may mean this inane hype might coming to an end now.
It's like going to school in a deep south county where they insist in teaching creationist bullshit. At what point is your education simply and plainly not lining up anymore with reality?
I would say pretty much immediately. If you teach lies, you are not teaching anymore and it stops being education. I
Re:Simple advice for the leaks (Score:4, Insightful)
The core reason for the hype is that some CEOs are having wet dreams of replacing workers with AI they don't have to pay. That's the whole engine driving his bullshit.
Mostly by CEOs who simply don't understand any of the whole deal and get blinded by greed.
Hey, I don't complain. The more these idiots fail, the more entertainment I will enjoy.
Re: (Score:3)
My take also. That this cannot work (no workers -> no wages -> no customers for your products) seems to be too difficult to understand for them. To be fair, that idea is probably more complex than anything taught in economics classes.
And yes, of course "AI" cannot deliver. Same as in all the previous AI hypes. What we may get is a bit better expert systems for very narrow tasks and and a bit better natural language interfaces. But that will be it. No "revolution" in sight anywhere except for the easil
Re: (Score:2)
Those "hallucinations" will get worse and worse with every generation, because every generation following will be trained on more and more AI generated content. Since we already have troubles telling AI generated from genuine human content, AI is prone to fucking up its derivative bullshit (sorry, "hallucinating"), AI is faster at generating content than humans and AI even being faster at generating content than you could curate it and detect the "hallucinations", how long do you think it will be until new
Re: (Score:2)
Indeed. Hence my statement that they may give us a bit better expert systems for very narrow tasks, because these can be trained on data that is known to be clean. Not cheap, needs a lot of manual work, but can be done.
Re: (Score:2)
I guess we'll soon run into the problem that AI is learning nearly exclusively from AI. Now, let's assume that 90% of AI generated content isn't total garbage. That means the first generation of AI had 100% sensible content to learn from (i.e. 100% human content). The next generation will have about 91%. The generation after hat, about 81%. and so on.
At what point will AI be reduced to learning more garbage than something that could actually be useful?
Oh, you're thinking small. It can get way worse than that, faster.
The latest fancy technique I've seen in use, to (presumably) reduce hallucinations, is using AI advisors as part of the training. Yup, they not only are using AI-generated training data, now they're using AI for evaluating the quality of the generated content as well. What could possibly go wrong? The theory behind this is that it can evaluate thousands of content samples much faster, greatly enlarging the volume of the training.
Allegedly t
Re: (Score:2)
The latest fancy technique I've seen in use, to (presumably) reduce hallucinations, is using AI advisors as part of the training.
This is typical in RLHF used in fine-tuning, not as part of the initial training, which I expect would be an absolute disaster. We train a reward model, on the basis of real human feedback, that can then be used to score responses from the base model. The reward model then takes the place of a human. The same technique has also been used to reduce 'hallucinations', though RLHF, as you might imagine, tends to have the exact opposite effect.
Believe it or not, this technique does seem improve the quality of t
Re: (Score:2)
Bruegel the Elder kinda predicted the whole mess in one of his paintings [wikipedia.org].
Re: (Score:2)
I suddenly feel like throwing in a load of Furry porn into the training.
I predict the outcome to be ... interesting, to say the least.
Re: (Score:2)
Re: (Score:2)
Exactly this, Don't give AIs confidential data, that's really a really dumb thing to do. Data in, data out, confidential data in, confidential data out.
Re: (Score:2)
Does it believe the hoaxes or not? (Score:2)
Given that there's a lot of hand-wringing about the data AI's are being trained on particularly with regards to social issues, one has to wonder if the humans are worried that Q isn't believing what it's being told and deciding that it's hallucinating instead.