How to Split a Song into Vocals and Accompaniment Locally for Free with StemDeck

Stem Separation Overview

Stem separation (also called "stem separation") reverses the mixing process by extracting individual instrument tracks—vocals, drums, bass, guitar, piano, and other—from a fully mixed audio file. Traditional filters and EQ cannot isolate these components cleanly because their frequency ranges overlap, so a deep‑neural‑network model is required.

StemDeck Tool

StemDeck is a free, open‑source desktop application that separates a song into up to six stems (vocals, drums, bass, guitar, piano, other). It accepts MP3, WAV, FLAC, OGG, MP4, M4A and YouTube URLs. The interface presents a multi‑track view similar to a DAW, allowing mute, solo, loop, waveform dragging, and export of selected stems. When a subset of stems is chosen, an additional "Original" track (the mix minus the selected stems) appears for A/B comparison.

Technical Architecture

The core separation engine is Meta’s open‑source Demucs model, specifically the htdemucs_6s version that outputs six stems.

Encoder : splits audio into short frames and encodes them into compact feature vectors.

Separation network : trained on paired full‑song and stem data to map mixed features to individual instrument components.

Decoder : reconstructs waveforms for the six output channels.

Processing pipeline:

Import audio (local file or YouTube URL)
↓ Decode & transcode with FFmpeg
↓ Demucs separates into 6 stems
↓ Render waveforms via Web Audio API & Canvas
↓ User interacts (mute, solo, loop, export)

Hardware detection automatically selects CUDA on NVIDIA GPUs, Metal Performance Shaders (MPS) on Apple silicon, or falls back to CPU.

Backend: FastAPI server exposing REST endpoints and Server‑Sent Events for real‑time progress updates. Frontend: plain JavaScript using the Web Audio API; waveforms are drawn on a <canvas>. The desktop wrapper is built with Tauri v2, producing native macOS and Windows binaries.

Additional analysis features include BPM detection with librosa, key/scale estimation, loudness measurement per ITU‑R BS.1770 using pyloudnorm, and peak sample calculation.

Installation and Usage

macOS: download the DMG, drag to Applications. The first launch downloads the Python runtime (~500 MB), FFmpeg, and the Demucs model (~170 MB). Windows: unzip the ZIP and run the executable; data are stored in ~/Documents/StemDeck by default and can be changed in settings.

Developer mode:

git clone https://github.com/stemdeckapp/stemdeck
cd stemdeck
./run.sh setup
./run.sh start

Docker images bundle a CUDA‑enabled PyTorch build for GPU acceleration; an Unraid package is also provided.

Typical UI workflow: drag an audio file or paste a YouTube link, press “Process”, watch the stages (Uploading → Analyzing → Separating → Mixing), then use shortcuts—M mute, S solo, Space play, [ / ] skip 5 seconds, L loop, I/O set loop in/out points—before exporting the chosen stems.

Comparison with Cloud Services

README comparison table lists the following limitations:

Model quality: open‑source Demucs is generally lower than proprietary models.

Missing extra features: no pitch‑shifting, chord detection, lyrics, or metronome.

No mobile apps for iOS/Android.

Single‑task processing only (one file at a time).

UI is functional but not polished to commercial standards.

Advantages are that the tool runs locally without account or upload, incurs no subscription fees, and can be used offline after the initial model download.

Target Users

Students and hobbyists extracting accompaniment or vocal tracks for practice.

Music producers and DJs sampling drums or bass locally.

Cover artists and remix enthusiasts needing isolated stems.

Users concerned about privacy who prefer not to upload audio.

It is less suitable for pipelines that require batch processing, mobile access, or the highest‑fidelity separation.

Design Philosophy

The project assembles existing open‑source components (Demucs, FFmpeg, yt‑dlp, librosa) rather than reinventing them, focusing on integration and user experience. The source repository is hosted at:

github.com/stemdeckapp/stemdeck

Images: