macOS 27 On-Device AI: Hands-On with fm CLI, Python SDK & Local API

The article documents a hands-on verification of macOS 27's on-device AI capabilities using a Mac mini M6 running macOS 27.0.1. The author confirms the presence of /usr/bin/fm and its man page, then walks through the required license acceptance via sudo fm license and checks availability with fm available, which returns "System model available".

Multiple Access Paths

Apple's Foundation Models framework provides two inference paths: a blue on-device path (SystemLanguageModel) that runs locally without internet after initial asset preparation, and an orange Private Cloud Compute (PCC) path that requires cloud connectivity. The local fm man page only lists the system model; the --model pcc option shown at WWDC26 is not present on this machine, so developers should not assume conference demos reflect current local availability.

CLI Verification

The fm --help output reveals commands: available, chat, count-tokens, license, respond, schema, and serve. The author tests interactive chat ( fm chat) showing a context window of 8,192 tokens (165 used in the session), single-shot generation with fm respond (including options for instructions, schema, no-stream, and transcript saving), and token counting ( fm count-tokens returns 11 for a sample Chinese sentence). A sandbox restriction error ( ModelManagerServices.ModelManagerError error 1008) appeared in an automated environment but succeeded outside the sandbox, indicating environment permissions matter.

File Input & Python SDK

Models do not automatically read files; the author provides a Python snippet that reads a text file and passes its content to fm respond via subprocess. For deeper integration, Apple's official apple-fm-sdk (imported as apple_fm_sdk) requires macOS 26+, Python 3.10+, Xcode 26+, and Apple Intelligence enabled. Installation in a virtual environment is recommended. The SDK supports structured generation via @fm.generable decorators and respond(generating=...). The author validates the structured-output concept using the CLI: fm schema object generates a JSON schema for a "Scene" object with narration, visual, and screen_text fields; fm respond --schema then produces valid JSON matching that schema. Hand-written JSON schemas were rejected with "Invalid schema", so using the tool-generated schema is advised.

Local HTTP API

The fm serve --host 127.0.0.1 --port 8765 command starts an OpenAI-compatible Chat Completions endpoint. The author verifies GET /v1/models returns the system model ID, and a non-streaming POST /v1/chat/completions request with model: "system" yields a Chinese response: "本地推理是指在设备本地进行计算，而不是将数据发送到云端进行处理。" Usage reports 63 input tokens, 29 output tokens, total 92. Both requests return HTTP 200. This provides a minimal integration path for existing HTTP clients, though streaming, image messages, and tool calls were not tested.

Practical Boundaries

For writing, the author uses the on-device model for material processing: extracting points, categorizing notes, compressing paragraphs, suggesting titles, and checking terminology consistency. It is not sufficient for research-heavy tasks requiring retrieval, source management, and fact-checking. Context length (8,192 tokens on this machine) must accommodate instructions, history, materials, and output; Chinese characters do not map 1:1 to tokens. A practical approach is thematic segmentation, point extraction, and outline assembly with source references preserved.

For video production, the model can convert verified articles into structured shot scripts (narration, visual description, on-screen text) but cannot generate video or audio. The author outlines a pipeline: model → structured script → TTS for narration → subtitle timing based on actual audio → rendering. Model-estimated durations are not reliable; final timing must come from generated audio. Code, numbers, and technical claims in the video must still be cross-checked against the source article.

Availability Caveats

Apple's documentation lists device, storage, language, and region requirements. As of the test date, devices purchased in mainland China cannot use Apple Intelligence; devices purchased abroad but used in mainland China with a mainland Apple Account are also restricted. The author's fm available and successful generations prove this specific Mac works in its test environment, but other machines must verify their own settings. On-device inference avoids per-token cloud costs but consumes local compute, memory, and battery; it is not an unlimited free server.

Next Steps

The author plans to start with small integrations: "short note point extraction" and "single-section article to shot script", validating against real materials for omissions, hallucinations, field errors, and latency before scaling into the production pipeline.