Upfront/TECH
Your Guide to the New Google
AI search results are not always trustworthy
IN JUNE, when people searched Google for the nonemergency police line in Salem, Oregon, the top search result was not the police station but the phone number for the husband of AI researcher Melanie Mitchell. “He was getting all these voicemails saying things like, ‘There’s somebody driving erratically on the highway,’ ” says Mitchell, a resident faculty member at the Santa Fe Institute.
Stories like this are not uncommon for Google’s AI Overview, which was launched widely in 2024 and is intended to give web searchers the information they seek in a summary at the top of the page; no need to click on a link. Sometimes it’s accurate. Other times, well… It did tell Google users that they could help cheese stick to pizza by using glue.
Why does this happen?
Google’s AI pulls information from its search results to create the Overview summary. It may find reliable information from reputable sources—or it may pull misinformation or confuse satire for facts. As in the nonemergency number mix-up, it may even “hallucinate,” piecing together a response that “sounds right, but it’s not correct,” says Chirag Shah, a professor at the University of Washington’s Information School.
In response to AARP questions about AI Overview, a Google representative replied: “At the scale of the web, with billions of queries coming in every day, there are bound to be some oddities, as there are with all search features. When issues arise ... we use those examples to improve our systems.”
What can you do?
Shah suggests treating AI Overview like a prototype: Approach it with caution. “We’re all the guinea pigs here,” he says.
And you can still scroll past AI Overview and go right to the web links. Or use a search engine without AI summaries, such as Dogpile or Startpage. —Lexi Pandell