Your Guide to the New Google Search Results

This past June, when people searched Google for the nonemergency police line in Salem, Oregon, the top search result was not the police station but instead the phone number for the husband of AI researcher Melanie Mitchell.

“He was getting all these voicemails saying things like, ‘There’s somebody driving erratically on the highway,’ ” says Mitchell, a resident faculty member at the Santa Fe Institute. Her husband’s phone number was not anywhere else on the internet, so as far as Mitchell can deduce, Google’s AI made up the number for the Salem police line.

Others have reported similar stories about Google’s AI Overview, which was launched widely in 2024 and is intended to give the web searcher the information they seek up top; no need to click on a link. In many cases, AI Overview presents a summary of information gleaned from its web search. Sometimes it’s accurate. At other times, well … it has told Google users that they can help cheese stick to pizza by using glue. Or worse, it has surfaced scam numbers for those looking for customer service information.

Why does this happen?

Upon receiving a search prompt, Google’s AI scrapes—or pulls information from—its search results. It may find reliable information from reputable sources—or it may pull in misinformation or confuse satire for facts. And there's more potential for error beyond bad search results. That’s because the AI isn’t simply regurgitating what it finds; it’s generating an original answer. Like in the case with the nonemergency number mix-up, it may even “hallucinate,” piecing together a response that “sounds right, but it’s not correct,” says Chirag Shah, a professor at the University of Washington’s Information School. In some instances, AI Overview may include links to its sources, but that’s not foolproof. “Actually that’s, in some respect, even worse, because it gives you a false sense of validity,” Shah says.

In response to AARP questions about AI Overview, a Google representative replied: “The accuracy rate for AI Overview is on par with other well-established search features.… At the scale of the web, with billions of queries coming in every day, there are bound to be some oddities, as there are with all search features. When issues arise—like if our features misinterpret web content or miss some context—we use those examples to improve our systems.”

Meanwhile, Google subsequently also released a search tool called AI Mode, in tandem with its Gemini AI technology. Though still in its experimental stages, AI Mode is available for consumers to try. According to Google, you can get deeper and more nuanced search results and ask any question with text, voice or images.