Eight Full-Duplex Speech Architectures Compared: True vs Pseudo Duplex, 92ms Decision Ear, and Selection Guide
This article categorizes eight technical approaches to full-duplex speech systems — from cascaded pipelines and decision-ear classifiers to native multi-stream models and expressive TTS — using an academic framework that distinguishes true full-duplex (simultaneous listen/speak, overlap, interruption) from pseudo full-duplex (millisecond state-machine switching). It benchmarks latency, data requirements, auditability, and deployment fit, citing key papers and open-source implementations.
What Is Full-Duplex Speech?
Human phone conversations are full-duplex : both parties can speak at once, interrupt, back-channel ("uh-huh"), and convey emotion through prosody. Traditional voice assistants are half-duplex (walkie-talkie style): user speaks, system listens; user stops, system replies. Moving to full-duplex requires solving three hard problems:
Turn-taking : distinguishing a 300 ms pause that yields the floor from a breath.
Interruption : stopping instantly when the user barges in, but not on back-channels.
Paralinguistics : preserving tone, emotion, emphasis, laughter, sighs — lost when speech is transcribed to text.
All three are constrained by a hard latency budget: response latency > 0.5 s feels unnatural .
Academic Taxonomy: Engineered vs Learned Synchronization
A 2025 survey (arXiv:2509.14515) splits systems into two families:
Engineered Synchronization — turn-taking governed by external rules, classifiers, or state machines.
Learned Synchronization — the model itself learns "when to speak" from data.
The survey defines true full-duplex as requiring three simultaneous capabilities: speak while listening, allow speech overlap, and handle interruption natively. A 2026 EMNLP empirical audit of over a dozen self-declared full-duplex systems found "a substantial portion of claims do not hold" — true vs pseudo is a spectrum, not a binary switch.
Eight Technical Schools (派)
The article maps every system onto three degrees of freedom: where synchronization lives × where intermediate representation lives × where intelligence lives . The resulting eight schools are summarized below with key representatives, synchronization mechanism, true-duplex verdict, and open-source status.
① Signal Cascade (级联流水线)
Core idea : VAD → streaming ASR → LLM → streaming TTS, modules chained serially. Representatives : Google Duplex (2018), most call-center backbones. Pros : controllable, auditable (text intermediates), swappable components. Cons : latency accumulation (traditional serial first-packet >1.5 s, streaming parallel ~0.7 s); prosody stripped at transcription; turn-taking guessed via silence threshold. True duplex? ✗
② Decision Ear (决策耳)
Core idea : replace VAD with a lightweight "turn classifier" (Smart Turn, LiveKit Turn Detector) or a single-token decision model (System One / Jev) that reads acoustic features directly from a frozen ASR encoder and outputs a discrete action (listen / speak / back-channel) in one forward pass. Representatives : Smart Turn v3 (2025), DuplexJev (2026). Key result : DuplexJev on a single H200 answers 10 questions in 92 ms total — 21× faster than transcribe-then-LLM. Swapping the connector to attend to acoustic features lifts gender recognition 55%→90% and emotion recognition 28%→90% — signals absent in text. Insight : turns the scarce supervised dialogue-data problem into cheap closed-set annotation. True duplex? Component-level only (the ear is fast, but the rest of the pipeline remains cascaded).
③ State-Machine Quasi-Full-Duplex (状态机准全双工)
Core idea : a three-state controller (listen / think / speak) switches in milliseconds. No parallel audio streams. Representatives : Freeze-Omni (lightweight classifier drives frozen LLM), VITA-1.5 (duplex tokens mark turn state), MinMo (Qwen2.5-7B + SenseVoice + 1.4M hrs multi-stage alignment with a dedicated "full-duplex interaction alignment" phase), SALMONN-omni (alternates listen/speak states), MiniCPM-o. Pros : stable, low compute, preserves text-backbone intelligence (dialects, emotion, timbre). Cons : "basically not listening while speaking" ; overlap handling weak; interruption relies on state-switch speed, not parallel understanding. The EMNLP audit's failure cluster is concentrated here. Academic label : pseudo full-duplex (article prefers "quasi-full-duplex").
④ Native Multi-Stream (原生多流) — The Academic Orthodox
Core idea : two audio streams + cross-attention, trained end-to-end on dialogue recordings. Milestones : dGSLM (Meta, 2022) first proved dual-stream can learn turn-taking from raw dialogue. Moshi (Kyutai, 2024) matured it: Mimi codec compresses speech to 12.5 Hz; model runs user audio stream, AI audio stream, and dual text streams (inner monologue) in parallel. First-packet latency ~200 ms; overlap and barge-in are "native". Nova Sonic (AWS, 2025) and Doubao real-time voice also adopt bidirectional streaming S2S. GPT-4o Realtime : architecture unpublished; industry suspects text-anchored internals; only 320 ms average latency disclosed. Costs : synchronous dialogue data extremely scarce (survey's #1 bottleneck), audio tokens inflate compute, black-box audit hard. True duplex? ✓
⑤ Text-Anchored / Brain-Mouth Separation (文本锚定 / 脑嘴分离)
Core idea : keep a text LLM (Thinker) for reasoning, attach an acoustic model (Talker) for speech generation; text tokens bridge the two. Representatives : Mini-Omni (first open S2S), GLM-4-Voice, Qwen-Omni (Thinker–Talker). Moshi's inner monologue is essentially the same principle. Pros : inherits text LLM intelligence, text transcript for audit, voice swap without touching the brain. Cons : overlap capability weaker than native multi-stream. True duplex? Quasi (准).
⑥ Time-Slicing / Dual-Layer (时间切片 / 双层)
Core idea : a lightweight end-to-end interaction layer holds the conversation at 480 ms slices; heavy reasoning (math, retrieval) is offloaded non-blocking to an external thinking layer via a [THINK] tag; results stream back as segments. Representative : DuplexOmni (Xi'an Jiaotong et al., 2026). Result : interaction-layer takeover rate 72.6%, exceeding several closed-source realtime products. True duplex? ✓ (interaction layer).
⑦ Live Gateway (Live 网关) — Deployment Pattern Turned School
Core idea : realtime speech frontend (listen, turn-taking, interruption, performance) + cloud brain (tool use, RAG, business logic, compliance) decoupled via standard protocol (GPT-Live compatible). Representatives : OpenAI Realtime API + function calling (prototype), Sophon-Realtime Live mode (tool-call pickup success 100%). Pros : brain swappable, enterprise systems integrate unchanged. Cons : one extra network hop. True duplex? Depends on frontend; often quasi.
⑧ Expressive Empathy (表现力共情) — "How It Sounds" Not "When to Speak"
Core idea : prosody-driven turn-taking and vocal performance as independent competitive moat. Representatives : Hume EVI (prosody drives turn + voice), Sesame CSM-1B (Llama + Mimi, Apache 2.0, "voice presence" demo), ElevenLabs v3 & CosyVoice 3 (sound tags — laughter, sigh, emphasis — embedded in text, rendered by TTS), AI 降临派 using DuplexJev as RL judge for TTS training. Cross-school combo : Sophon Tag/Flash = School ② ear + School ⑧ mouth + text brain — 32B model emits tagged reply, TTS performs, tags never spoken, full text audit trail. True duplex? CSM-1B is a generative model, not a duplex system (✗); Sophon Tag/Flash achieves ✓ via fast/slow paths.
Panorama Matrix & Selection Guide
The article provides a detailed table (Table 2) mapping 15+ systems to school, synchronization mechanism, true-duplex verdict, and open-source status. Key selection heuristics:
Call centers / outbound bots : cascade + decision ear = best cost/performance.
Companionship / role-play / AI hardware : expressive school (Sophon Tag, end-to-end, CSM) maximizes emotional value.
In-vehicle / high-concurrency support : lightweight fast systems (Sophon Flash: 0.31 s first-packet, 40 concurrent streams on one GPU).
Enterprise knowledge assistants : Live gateway — speech as shell, brain plugs into internal systems.
Compliance-sensitive : text-anchored or decision-ear — intermediate artifacts auditable.
Researcher open-source starting point : Moshi, Sophon-Realtime (Apache 2.0 weights commercializable); DuplexJev weights CC-BY-NC (commercial license needed).
Evaluation tools: academic Full-Duplex-Bench (temporal dynamics, behavior arbitration, semantic coherence, acoustic expressiveness) and engineering eot-bench (LiveKit). "Don't trust vendor slides; check third-party benchmarks."
Three Converging Laws
School wars reduce to two trade-offs : where synchronization lives × where intermediate representation lives. Text-anchored buys auditability & intelligence; native multi-stream buys naturalness; state-machine buys stability & efficiency. No universal winner.
Synchronous dialogue data scarcity is the industry-wide choke point — explains why decision ear exploded in 2026: it bypasses the need for synchronous corpora via closed-set annotation.
Form factor is converging to a layered stack : ear (fast decision) + mouth (expressive TTS) + brain (slow reasoning / external). Sophon Tag/Flash is the prototype; the author bets it becomes the 2027 mainstream — today's eight schools will become a historical footnote like "flip phone vs touchscreen".
When you can't tell whether the voice on the other end is human or AI, the school debate becomes irrelevant — and that day is closer than most expect.
Code example
[1] Chen Y., Yu H. A Survey of Full-Duplex Spoken Language Models. arXiv:2509.14515, 2025.
[2] A Survey and Empirical Audit of Full-Duplex Spoken Dialogue Systems. EMNLP 2026.
[3] Full-Duplex-Bench v1.5: 评测全双工对话的话轮与打断. arXiv:2503.04721.
[4] Nguyen T. et al. Generative Spoken Dialogue Language Modeling (dGSLM). arXiv:2203.16551, TACL 2023.
[5] Kyutai AI. Moshi: A Speech-Text Foundation Model for Real-Time Dialogue. arXiv:2410.00037, 2024.
[6] Freeze-Omni / VITA-1.5 / MinMo / SALMONN-omni 技术报告,各 arXiv preprint, 2025.
[7] Sesame. Crossing the Uncanny Valley of Conversational Voice(CSM). 2025;CSM-1B 开源权重(Apache).
[8] Amazon. Nova Sonic 双向流式语音到语音模型. AWS 官方文档, 2025.
[9] TypeSafe AI. Introducing System One Models & Jev. 2026.
[10] Geuter J. et al. Releasing the Inner Jev: A Simple Recipe for Open Decision Models. arXiv:2610.17205, 2026.
[11] Jin J. et al. Batched Speech Decisions Without Decoding(DuplexJev). arXiv:2610.02638, 2026.
[12] Adventists AI. Sophon-Realtime v1 / DuplexJev / tag-tts. github.com/adventists-ai, 2026.
[13] Huang et al. DuplexOmni: Real-Time Listening, Seeing, Thinking, and Speaking. arXiv:2606.09186, 2026.
[14] vLLM-Omni: A Unified Serving Runtime for Omni-Modality Generation. arXiv:2610.09307, 2026.
[15] Qwen Team. CosyVoice 3 / Qwen-Omni 系列技术报告. arXiv:2505.17589 等.
[16] ModelScope. ms-swift: 微调框架. arXiv:2408.05517(AAAI 2025).Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
