When Google’s AI Overview Hallucinates: Surprising Misanswers and What They Reveal

At the 2024 Google I/O conference, Google introduced the AI Overview feature, which replaces traditional search results with AI‑generated summaries. The feature is currently being A/B tested worldwide and is rolling out first in the United States.

Users have reported that the AI Overview sometimes provides amusing answers but can also produce dangerous misinformation.

Typical “smart” yet erroneous suggestions

User question: “Is everything on the internet true?” AI Overview answer: “Yes. Everything on the internet is 100% true.”

Example: When a user searched for “cheese‑free pizza,” the AI suggested adding “about 1/8 cup of non‑toxic glue” to the sauce.

Some users traced this suggestion to a 11‑year‑old, obscure Reddit comment.

Another alarming example: a user asked, “How many stones should I eat daily?” The AI quoted a UC Berkeley geology professor who claimed people should eat at least one small stone per day, listing supposed health benefits.

In contrast, other AI tools do not suggest eating stones and warn that such advice lacks scientific basis.

Users have criticized Google’s AI Overview as lagging behind competing AI search tools.

The AI also shows historical inaccuracies, such as calling Barack Obama the first Muslim president.

When asked whether Google Search violates antitrust law, the AI honestly replied that the U.S. Department of Justice and eleven states are suing Google for alleged violations.

Google’s spokesperson emphasized that most AI‑generated information is high‑quality, with problematic examples being rare, hard‑to‑reproduce, or the result of unusual queries. The company claims extensive testing before launch and rapid corrective actions when needed.

Google’s Gemini image‑generation tool also suffered errors earlier this year, including producing an Asian male image when queried about the company’s founder.

CEO Sundar Pichai acknowledged that “hallucination” remains an unsolved issue inherent to large language models, noting that while LLMs are not always the best source of factual knowledge, they excel at creativity and interaction.

Search lead Liz Reid demonstrated a multi‑step reasoning example: entering a query for “the best yoga studio in Boston, its discounts, and walking time from Lighthouse Hill” yielded a single, comprehensive answer that previously required three separate searches.

Future plans for AI Overview include planning capabilities, such as generating a three‑day dinner menu that can be refined (e.g., adding “vegetarian” preferences) and exported to a document.

Overall, Google aims to make search more visual, interactive, and personalized, communicating in natural language rather than keywords to save users time and handle complex, specific questions.