What Does Galileo’s New Hallucination Index Reveal About Today’s Top Generative AI Models?

Artificial‑intelligence company Galileo has just released its latest Hallucination Index, a framework that evaluates 22 leading generative AI large models.

The index uses a metric called “contextual adherence” to measure closed‑domain hallucinations, i.e., cases where a model generates content not provided in the given context.

Ranking Results

According to the rankings, the overall best RAG (retrieval‑augmented generation) model is Anthropic’s Claude 3.5 Sonnet, which scores near‑perfect and surpasses last year’s winning closed‑source model from OpenAI.

From a cost perspective, Google’s Gemini 1.5 Flash offers the best performance‑per‑dollar.

Alibaba’s Qwen2‑72B‑Instruct is the top open‑source model overall, while Meta’s llama‑3‑60b‑instruct leads in short‑context RAG tests.

When broken down by context length:

Short‑context RAG: best closed‑source model is Claude 3.5 Sonnet.

Mid‑context RAG: best closed‑source model is Google’s Gemini‑1.5‑flash‑001, with cost being the decisive factor.

Large‑context RAG: Claude 3.5 Sonnet again takes the top spot.

CEO Insight

Galileo CEO and co‑founder Vikram Chatterji explains that today’s fast‑moving AI landscape forces developers and enterprises to balance cost, accuracy, and reliability while leveraging generative AI. Existing benchmarks often reflect academic use cases rather than real‑world applications.

The new index aims to test models in practical scenarios that require LLM‑driven data retrieval—common in enterprise AI deployments—providing teams with actionable data to choose the right model at the right price for the right task.

About Claude 3.5 Sonnet

Anthropic’s June‑released Claude 3.5 Sonnet competes directly with OpenAI’s GPT‑4o and Google’s Gemini 1.5. It offers notable improvements in performance, speed, and cost‑efficiency, excelling in text and image analysis, code generation, and multi‑step workflows.

Anthropic claims the model surpasses its predecessor Claude 3 Opus and outperforms leading models such as GPT‑4o and Gemini 1.5 on multiple benchmarks, while also demonstrating a better grasp of humor and more human‑like writing style.