Hangzhou Startup’s AI Music Model Claims to Truly Understand Chinese Lyrics

Industry Landscape

The AI music sector is booming yet contentious, with massive capital inflows exemplified by Suno’s $400 million Series D round and ongoing copyright lawsuits involving major labels. Domestic players such as ByteDance, Tencent, NetEase, Kunlun, MiniMax, and numerous vertical startups are all vying for market share.

Company Background and Model Launch

Hangzhou‑based Gege Music, founded in 2019 by Long Yong, has pursued a long‑term AI music strategy. Leveraging team experience from NetEase and Alibaba’s music divisions, the company announced the full‑stack, self‑developed "Gege AI Music Model" and completed its version iteration for public release.

Model Characteristics

The model emphasizes localized Chinese music creation, delivering natural vocals, nuanced emotional expression, and genre styles (Mandopop, guofeng, folk) that surpass generic models. It generates a full 3‑minute stereo track in roughly 10 seconds on a single H‑series GPU, outputting separate vocal and accompaniment stems for low‑cost commercial use.

Training Pipeline

Training proceeds in three stages: (1) a VAE learns basic audio compression and reconstruction; (2) a billion‑parameter diffusion backbone is pretrained on a fully licensed Chinese song corpus; (3) preference‑aligned optimization (music‑specific DPO) aligns outputs with Chinese listeners’ aesthetic preferences.

Technical Innovations

Key innovations include a "phoneme‑timeframe soft‑alignment prior" that injects explicit timing information for each Chinese character via attention bias, addressing the notorious Chinese pronunciation problem. A dual‑stream independent generation architecture separates vocal and accompaniment generation, using cross‑stream attention to maintain rhythmic and harmonic coherence.

The model employs a hierarchical multi‑dimensional conditioning system: global style attributes (emotion, genre, key) are modulated through AdaLN‑Zero, while lyrics and melody are aligned frame‑by‑frame via cross‑attention. Independent control over condition intensities lets creators balance lyric adherence versus melodic freedom. A cross‑modal encoder shares a unified semantic embedding space for text prompts and audio references.

Efficiency is achieved through a non‑autoregressive parallel generation framework that denoises the latent representation of the entire song simultaneously, yielding a real‑time factor of ~0.05 (20× faster than playback). Additional optimizations such as few‑step sampling and model quantization keep inference costs minimal, and a chunk‑wise continuation mechanism enables streaming generation.

Copyright Partnership and Business Model

Gege AI signed a non‑exclusive music copyright revenue‑sharing agreement with ByteDance. Generated songs, recordings, and music videos are fully compliant and distributed across ByteDance’s ecosystem (Douyin, Jianying, etc.). Revenue from subscriptions, digital sales, and ad sharing is split per contract, and newly generated non‑exclusive tracks automatically enter the licensed catalog, creating a closed loop from self‑trained model to clear copyright to monetization.

Folk Music Initiative

Recognizing the lack of authentic Chinese folk music in existing datasets, Gege AI launched a nationwide field‑recording campaign, capturing raw instrument sounds, regional tunes, and opera vocal styles. After securing copyright, these recordings become exclusive training data for a dedicated "Gege AI Folk Music Model," planned in three phases: building a folk sound library, fine‑tuning the generation pipeline for solo, ensemble, and vocal folk pieces, and finally launching a folk‑creation portal for users.

Conclusion

The article notes that while AI‑generated music is reshaping content creation and monetization, the primary bottleneck in China remains copyright clarity. Gege AI’s end‑to‑end approach—zero‑shot pretraining on fully licensed Chinese corpora, technical innovations for Chinese phonetics, and strategic partnership for distribution—offers a viable pathway to commercialize AI music that truly resonates with Chinese cultural and linguistic nuances.