How a Single /watch Command Lets Claude Actually See Video

Bridging Claude Code’s Video Blind Spot

Claude Code can read web pages, run scripts, and clone repositories, but it cannot understand video content—it either guesses from titles or receives incomplete subtitles, losing most visual information. The bradautomates/claude-video repository fills this gap by attaching a "eye" to Claude and other compatible agents via a single /watch command.

Five‑Step Processing Pipeline

Accept a URL (YouTube, Loom, TikTok, Instagram, etc. supported by yt‑dlp) or a local video file path.

Prefer native subtitles; if unavailable, download a mono 16 kHz/64 kbps audio track (~480 KB/min) and run Whisper. The default Whisper model is whisper‑large‑v3 (cheap and fast), with whisper‑1 as a fallback.

Extract frames with ffmpeg. The default efficient mode grabs only key frames (~0.5 s per frame). Alternative modes are balanced and token‑burner, which use scene‑change detection.

Deduplicate frames using a pure‑stdlib Python script that compares the average brightness of adjacent 16×16 grayscale thumbnails; a threshold of 2.0 discards near‑identical frames, preventing static screen‑record segments from consuming the frame budget.

Pass each frame’s path (annotated with t=MM:SS timestamps) and the subtitle file to Claude via the built‑in Read tool, which reads JPEG images into the model’s context.

Usage Examples

/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?
/watch bug-repro.mov when does the UI break?
/watch "$URL" --start 2:15 --end 2:45

Real‑World Scenarios from the README

Dissect viral video hooks :

/watch https://youtu.be/<viral-video> what hook did they open with?

shows the opening frames and subtitles.

Debug from screen recordings : /watch bug-repro.mov what's going wrong? pinpoints the failing frame.

Strip hype from launch videos :

/watch <launch-video> what's actually new — skip the hype

extracts genuine product announcements.

Turn playlists into searchable notes : combine with prompts like summarize this to a note to convert a YouTube course into searchable snippets.

Design Insight: Frame Budget

Video understanding is dominated by image token cost. Instead of a fixed frame‑every‑N‑seconds rule, the script allocates frames based on total video length:

≤30 s → ~30 frames (covers almost every key point)

30 s‑1 min → ~40 frames (still dense)

1‑3 min → ~60 frames (comfortable)

3‑10 min → ~80 frames (sparse but usable)

>10 min → capped at 100 frames; a "sparse scan" warning suggests focusing on a segment.

Benchmark Results (49‑minute, 720p YouTube recording)

transcript

mode (no frames): 0 frames, ~4.5 s processing, 0 image tokens (subtitle ≈26.6 k text tokens). efficient mode (key frames): 50 frames, ~0.5 s extraction, ≈9.8 k image tokens – 40× faster. balanced mode (scene‑change detection): 100 frames, ~20.9 s, ≈19.7 k image tokens. token‑burner mode (unlimited scene detection): 116 frames, ~21.0 s, ≈22.8 k image tokens.

The numbers illustrate the trade‑off: efficient is speed‑focused, token‑burner maximizes visual detail, while balanced fits most everyday needs.

Installation & First‑Run Setup

Three host environments have dedicated installation commands:

Claude Code (recommended) :

/plugin marketplace add bradautomates/claude-video /plugin install watch@claude-video

Generic CLI clients (Codex, Cursor, Copilot, Gemini, etc.) : npx skills add bradautomates/claude-video -g Use -g for global installation or omit it for project‑local placement.

Claude.ai web UI : download watch.skill from the release page, drag it into Settings → Capabilities → Skills, and enable "Code execution and file creation".

The first invocation runs setup.py --check, which on macOS automatically executes brew install ffmpeg yt‑dlp; Linux/Windows print the appropriate commands. Whisper API keys are optional and stored in ~/.config/watch/.env with 0600 permissions.

Known Limitations

API key is unnecessary for videos that already provide native subtitles; Whisper is only used for local files, TikTok, some Vimeo videos, or YouTube videos lacking subtitles.

Videos longer than ten minutes become sparse in efficient or balanced modes; the script warns and suggests focusing on a segment or using --detail token-burner to lift the cap.

Deduplication relies on a simple brightness‑difference threshold, so any change in a single line of code, a terminal scroll, or a solid‑color slide is treated as a distinct frame—an intentional design choice to avoid accidental loss.

Who Should Use It

Anyone who frequently hands YouTube videos or screen recordings to an AI assistant for product‑launch analysis, competitor research, or content extraction.

Developers who receive bug‑repro videos and want the agent to locate the failing frame automatically.

Users of Claude Code, Codex, Cursor, or similar agents who need video context in long workflows.

People who want to convert YouTube courses or channel series into searchable notes.

It is not suited for occasional video viewers; installing the tool would be overkill.

Getting the Repository

GitHub address: github.com/bradautomates/claude-video