Build a Fully Private Ubuntu AI Assistant with DeepSeek‑R1 and AnythingLLM (No Internet Needed)

Why Ubuntu + RAG?

RAG (Retrieval‑Augmented Generation) adds a local document library that the model queries before generating a response.

Ubuntu advantage — native support for NVIDIA CUDA and AMD ROCm drivers and the most extensive Docker and Python ecosystems for AI development.

Prerequisites

Ubuntu 22.04 or 24.04 LTS

NVIDIA GPU with 8 GB+ VRAM recommended (Apple M‑series or CPU‑only also works, slower)

Step 1 — Deploy Ollama

Install Ollama with a single command: curl -fsSL https://ollama.com/install.sh | sh Pull the DeepSeek‑R1 model that matches the available GPU memory:

# 8 GB VRAM – use 7b/8b version
ollama run deepseek-r1:7b

# Smaller VRAM or CPU‑only – use 1.5b
ollama run deepseek-r1:1.5b

When the >>> prompt appears, type Who are you?. A fluent answer confirms the local model is running.

Step 2 — Build the knowledge base with AnythingLLM

AnythingLLM is an open‑source full‑stack RAG solution that provides a desktop AppImage for Ubuntu.

Download the Linux AppImage from the following URL (plain text): https://anythingllm.com/desktop

Run the AppImage:

chmod +x AnythingLLMDesktop.AppImage
./AnythingLLMDesktop.AppImage

Configure in the UI:

LLM Provider: Ollama

Ollama URL: http://127.0.0.1:11434 Model: deepseek-r1:7b (or the version installed)

Vector Database: default LanceDB (built‑in, no extra install)

Step 3 — Feed the data

Create a new Workspace (e.g., “My Secrets”).

Click the upload button and drag‑drop PDFs, Markdown files, Word documents, etc.

Press “Move to Workspace” → “Save and Embed”.

The system slices, vectorises, and stores the documents locally; no bytes are sent to the internet.

Step 4 — Offline test

Physically disconnect the network cable or disable Wi‑Fi.

Ask a question that exists only in the uploaded files, for example:

“What is the budget for the living‑room tiles in my 2026 renovation plan? How much can I save by not buying imported brands?”

The AI returns the exact numbers from the PDF and provides reasoning based on the document content.

Conclusion

Data sovereignty : all documents remain on the local disk.

Zero cost : every component (Ollama, DeepSeek‑R1, AnythingLLM, LanceDB) is open‑source and free.

High performance : DeepSeek‑R1 delivers strong inference speed, making a private knowledge base practical.