How Cactus Turns Any Smartphone into a Powerful Offline AI Assistant

Overview

Cactus is an open‑source, lightweight framework that enables large language models to run entirely on mobile devices without network connectivity. It supports chat, visual language model (VLM) image recognition, text‑to‑speech (TTS), and text embeddings, and provides cross‑platform bindings for Flutter, React‑Native, and C++.

Key Technical Benefits

Zero‑dependency local execution : Models are loaded and inferred on‑device, eliminating the need for Wi‑Fi or cellular data. In benchmark tests a 100‑word generation takes ~0.5 s locally versus ~2 s in the cloud (≈30 % faster).

Low resource consumption : Optimized loading allows 4 B‑parameter models (e.g., Qwen3‑4B) to run on older phones such as iPhone 13 or Xiaomi 13 with ~40 % less memory than competing tools.

Full multimodal support : Single engine provides chat, image recognition, TTS, and embedding extraction.

Cross‑platform adapters : Flutter, React‑Native, and C++ APIs let developers embed the engine in iOS/Android apps or native applications.

Open‑source and privacy‑first : All source code is public; any GGUF‑format model (Llama, Gemma, Qwen, etc.) can be imported, keeping data on the device.

Low‑power design : Continuous one‑hour AI chat consumes less battery than short‑video playback on the same device.

Representative Use Cases

1. Offline note‑taking

In a subway with no network, a user can invoke the local chat with a prompt such as “Summarize Q2 priorities: user growth, product iteration, channel expansion”. The model returns a structured note in under one second, which can be copied to any notes app.

2. Offline visual guide

Capture a photo of a landmark.

The VLM identifies the object (e.g., “Zhuozheng Pavilion, Ming‑dynasty architecture”).

The description is converted to speech via TTS (female voice) and can be followed by follow‑up queries like “What local snacks are recommended?”

3. Extending the life of older phones

On an iPhone 13 the Gemma‑3 1 B Q4 model loads without lag, enabling:

One‑second generation of a professional leave‑request email in English.

Recognition of tabular data from a photographed Excel sheet and automatic textual summarisation.

Conversion of the summary to speech for hands‑free review while driving.

Getting Started

Path 1 – End‑user demo app (≈1 minute)

Download the binary for your platform:

iOS – IPA from GitHub Releases, install via AltStore or TestFlight.

Android – APK (Android 10+), install directly.

Place a GGUF model file (e.g., gemma-3-1b-q4.gguf) in the app‑specified directory.

Launch the app, select the model, and use chat, image recognition, or TTS without further configuration.

Path 2 – Developer integration (Flutter example)

Add the dependency to pubspec.yaml:

dependencies:
  flutter:
    sdk: flutter
  cactus: ^0.1.2

Install the package:

flutter pub get

Load a model and generate a response:

import 'package:cactus/cactus.dart';

// Initialise the model
final context = await CactusContext.init(CactusInitParams(
  modelPath: '/path/to/model.gguf', // absolute path on the device
  contextSize: 2048,                // token context length
  threads: 4,                       // number of CPU threads to use
));

// Generate a chat completion
final result = await context.completion(CactusCompletionParams(
  messages: [ChatMessage(role: 'user', content: 'Help me write a product tagline')],
  maxPredictedTokens: 50,
  temperature: 0.7,
));

print(result.text); // prints the generated tagline
context.free(); // release native resources

This integration removes the need for cloud API calls, reducing cost dramatically (e.g., 1 million cloud calls costing ~US$10 k become effectively free on‑device).

Project Repository

https://github.com/cactus-compute/cactus