Can I Trust Google’s AI Search Results?

You may already see these summaries appear on top of news and other search results, though they don’t show up for every query or on every device or browser.

When I recently asked Google, “Why is the sky blue?” AI Overviews explained, “Rayleigh scattering,” a process that occurs when sunlight passes through the Earth’s atmosphere and is scattered by tiny air molecules.

Smack in the middle of this explanation were bubbles I could click to read more from such sources as NOAA SciJinks and National Geographic Kids. Just below AI Overviews are the familiar blue Google search links.

I’m no expert but felt confident in this AI summary.

AI still can’t tell every fact from fiction

But that confidence isn’t always earned. Shortly after I/O, Google was forced to play defense.

Some AI Overviews results that went viral on social media were downright wacky or even dangerous. A result suggested that you should add nontoxic glue to keep cheese from sliding off a pizza, seriously. Want a little Elmer’s with your marinara sauce?

Another answer counseled people to ingest a rock a day because they contain minerals and vitamins that are supposed to help with digestion. The, um, rocky advice was traced to the humor site The Onion, the pizza example to an old Reddit post.

Google search executive Liz Reid conceded in a blog that “some odd, inaccurate or unhelpful AI Overviews certainly did show up,” and admitted Google needed to do a better job in its “ability to interpret nonsensical queries and satirical content.”

Among the fixes, Reid blogged Google would “limit the use of user-generated content that could offer misleading advice.” She added that Google would not display AI Overviews for “hard news topics, where freshness and factuality are important.”

She also went to pains to explain that AI Overviews work differently from AI chatbots that solely rely on what techies refer to as large language models, or LLMs, essentially vast reservoirs of training data. While AI Overviews tap into customized language models as well, they also tie into the web ranking systems Google uses for traditional search.

So where does this leave you?

As with anything you come across on the internet, apply a gut check. Google has a reason to slap a “Generative AI is experimental” disclaimer next to AI Overviews. So if something seems weird or off, it probably is.

Verify the facts by visiting sites you trust, and by no means take what you read in AI Overviews as gospel.

Faced with embarrassing feedback and lessons learned, Google’s AI Overviews are bound to become more reliable. But we’re not there yet.

As I was writing this column a colleague stumbled across an AI Overviews that cited ARP’s Social Security for Dummies as a resource. Especially around here, we know that a letter was missing. It’s AARP.

Bonus tip: Should I turn on Google’s ‘Enhanced safe browsing’?

Google’s Enhanced safe browsing tool is a double-edged sword. The feature works in the background in Chrome and Gmail to check for risky web addresses, downloads and passwords exposed in a breach. That’s a good thing.

The drawback? You will share even more browsing data with Google than you already do.

If you’re comfortable with the trade-off, click on your Google Account in Chrome. Go to Settings ⋮ | Manage Your Google Account | Security. Scroll down to Enhanced Safe Browsing for your account. Click the Off button | Enhanced Browsing.

Google will show you a message outlining the feature. Click Turn on or, if you change your mind, No thanks.