Running Large Language Models on Raspberry Pi with Ollama: A Step‑by‑Step Guide

Installing Ollama on Raspberry Pi (and other Linux)

To install Ollama on a Raspberry Pi OS, download the official script and execute it in a single command:

curl https://ollama.ai/install.sh | sh

Exploring Different Large Language Models

After installation, you can run various LLMs directly on the Pi. The following hardware guidelines apply: at least 8 GB RAM for a 7B model, 16 GB for a 13B model, and 32 GB for a 33B model.

TinyLlama

TinyLlama, with 1.1 B parameters, is an ideal first model for local inference. ollama run tinyllama Example query: “What is the use case of the div tag in HTML?” The model returns a concise answer (see image).

Phi

Phi is a 2.7 B parameter model that also runs on the Pi. ollama run phi Example query: “What is the difference between a network switch and a hub?” The response is shown in the image below.

LLaVA

LLaVA is a 7 B parameter multimodal model capable of image description. ollama run llava Downloading the model (~3.9 GB) takes several minutes. An example query asks the model to describe a cat image stored at /media/hdd/shared/test.jpg. The process was aborted after 30 minutes due to long response time, but the partial answer demonstrates accurate visual understanding.

Conclusion

Combining Raspberry Pi 5 with Ollama provides a solid foundation for developers and AI enthusiasts who want to run open‑source language models locally, pushing the limits of edge computing and opening up many experimental possibilities.