Google's 'AI Overview' Pointed Him to a Customer Service Number. It Was a Scam

A real estate developer searched Google for a cruise ship company's customer service number, reports the Washington Post, calling the number in Google's AI Overview. "He chatted with a knowledgeable representative and provided his credit card details," the Post's reporter notes — but the next day he "saw fishy credit card charges and realized that he'd been fooled by an impostor for Royal Caribbean customer service."

And the Post's reporter found the same phone number "appearing to impersonate other cruise company hotlines and popping up in Google and ChatGPT" (including Disney and Carnival's Princess line): He'd encountered an apparent AI twist on a classic scam targeting travelers and others searching Google for customer help lines of airlines and other businesses... The rep knew the cost and pickup locations for Royal Caribbean shuttles in Venice. [And "had persuasive explanations" when questioned about paying certain fees and gratuities.] The rep offered to waive the shuttle fees...

Here's how a scam like this typically works: Bad guys write on online review sites, message boards and other websites claiming that a number they control belongs to a company's customer service center. When you search Google, its technology looks for clues to relevant and credible information, including online advice. If scammer-controlled numbers are repeated as truth often enough online, Google may suggest them to people searching for a business.

Google is a patsy for scammers — and we're the ultimate victims. Google's AI Overviews and OpenAI's ChatGPT may use similar clues as Google's search engine to spit out information gleaned from the web. That makes them new AI patsies for the old impostor number scams.
"I've seen so many versions of similar trickery targeting Google users that I largely blame the company for not doing enough to safeguard its essential gateway to information," the reporter concludes, (adding "So did two experts in Google's inner workings.") The Post is now advising its reader to "be suspicious of phone numbers in Google results or in chatbots."

Reached for comment, a Google spokesman told the Post they'd "taken action" on several impostor numbers identified by the reporter. That spokesman also said Google continues to "work on broader improvements" to "address rarer queries like these." OpenAI said that many of the webpages that ChatGPT referenced with the bogus cruise number appear to have been removed, and that it can take time for its information to update "after abusive content is removed at the source."
Meanwhile, the man with the bogus charges has now canceled his credit card, the Post reports, with the charges being reversed. Reflecting on his experience, he tells the Post's readers "I can't believe that I fell for it. Be careful."

Re:ok?

[Z00L00K]

The AI is more wrong than right in many searches that should be simple so I have started to totally ignore those results and not even do a thumbs down because it's not worth my time to teach that AI what it does right or wrong.

Re: ok?

[blue trane]

Is my experience that AI is only wrong 10% of the time (e.g. when it regurgitates mainstream economic theories) unique?

Why shouldn't the scammed check the number first? Why does that reporter excuse laziness in not doing the slightest bit of due diligence?

Re: ok?

[aurasdoom]

Check the number where? I'm Google? With ChatGPT? The same ones that told them the number?

We use Google to check the phone numbers!

Re:

[ebunga]

But "everyone" says it's "getting better" and doing it "very very fast" and that I am very smart and that the AI is a god. All hail the AI.

Anything travel related should be suspect

[damn_registrars]

I travel a fair bit for work, and if I'm going to a new (to me) city I'll often google "X hotel chain Y city" to see what options I have. Unfortunately Google has been victimized in their search results on this for a long time; it's not just redirects to various third-party hotel sites but it can be other sites entirely as well. This can lead to outdated or completely inaccurate information, and bad phone numbers too. Sometimes I'll see the correct link about 5-6 entries down, but that's not guaranteed -

Re: Anything travel related should be suspect

[blue trane]

Why can't you get with the program, and blame AI, which is accelerating climate change?

Well duh

[locater16]

Google was already SEO'd to absolute shit for search, why shouldn't the same happen in record time for AI?

AI is getting better, but

[MpVpRb]

Always cross check everything
This is why I like Perplexity. It gives me links to sources

Could Google use AI for fraud detection?

[misnohmer]

AI should be able to "investigate" any phone number it's going to be publishing in Google results. Check who is the owner of said number, does it match the company's website, heck, have AI call the number to attempt to verify it (perhaps ask the operator who picks up to provide a callback number/extension which would match up with any of the official company numbers). Sure, some companies with shitty IT would get flagged as fake, but if Google flags their number as fake, they'll be fixing it in no time - no