How Chrome Canary Lets You Run Gemini Nano LLM Directly in the Browser

Modern AI tools such as ChatGPT, Claude, and Gemini are typically accessed via server‑side APIs, which requires downloading large models or relying on external services. Google is developing a set of web platform APIs and browser features that allow large language models (LLMs) to run directly in the browser.

Gemini Nano

Gemini Nano is the first experimental model in Google’s Gemini series, designed for efficient local execution on most modern desktops and laptops. It enables web sites or web apps to provide AI functionality without deploying or managing their own AI models.

Easy deployment: the browser handles device capability detection and model updates, eliminating the need for developers to manage large model downloads, storage constraints, or runtime memory budgeting.

Hardware acceleration: the built‑in AI runtime leverages available GPU, NPU, or falls back to CPU, delivering optimal performance on each device.

Offline use: the model can run without an internet connection.

These advantages also avoid the security and privacy concerns of third‑party AI services.

Enabling Gemini Nano in Chrome Canary

Google is testing Gemini Nano on Chrome Canary. To try it, install Chrome Canary from https://www.google.com/chrome/canary/ and enable the Prompt API for Gemini Nano via the following flags:

chrome://flags/#prompt-api-for-gemini-nano

chrome://flags/#optimization-guide-on-device-model

After enabling the flags, restart Chrome. The model (under 3 GB) may require up to 22 GB of free disk space to download.

Use the canCreateTextSession API to check the model’s readiness; it returns "no", "after-download", or "readily". The download status can be verified in chrome://components under the version 2024.6.5.2205 of the Optimization Guide On‑Device Model.

Prompt API Overview

Inspecting window.ai reveals the available methods. The Prompt API, proposed for Chrome, aims to provide a unified JavaScript API for web developers to access browser‑provided language models, abstracting tokenization, system messages, and handling failures gracefully. It supports both on‑device and cloud‑based implementations.

In Chrome Canary, the Gemini Nano API follows this proposal. The primary callable method is createTextSession, which offers two output modes:

Full output

const session = await ai.createTextSession();
const result = await session.prompt("Where is the capital of China?");
console.log(result);

Streaming output

const stream = await session.promptStreaming("Introduce the major internet companies in Beijing?");
for await (const chunk of stream) {
  console.log(chunk);
}

Demo: In‑page Text Translation

The built‑in AI can power many web scenarios, such as text classification, content summarization, title generation, Q&A, translation, and proofreading. Below is a simple demo that translates selected English text to Chinese using the Prompt API.

document.addEventListener('mouseup', async function() {
  let selectedText = window.getSelection().toString().trim();
  if (selectedText && /[a-zA-Z]/.test(selectedText)) {
    try {
      console.log(`Translating: ${selectedText}`);
      const translatedText = await translateText(selectedText);
      console.log(`Result: ${translatedText}`);
    } catch (error) {
      console.error('Translation error:', error);
    }
  }
});

async function translateText(text) {
  const session = await ai.createTextSession();
  const prompt = `Translate the following English text to Chinese: "${text}"`;
  const result = await session.prompt(prompt);
  return result;
}

The demo shows how a few lines of JavaScript can leverage the browser’s AI capabilities without any external service.

Developers are encouraged to experiment with these APIs and hope for broader browser support in future stable releases.