Building a Reliable Live‑Streaming Host Assistant: Harness Engineering Practices for the Taobao Agent

Why a Harness Is Needed for a Live‑Streaming Host Agent

Large‑scale host agents such as Taobao’s live‑streaming assistant must act instantly, incur real monetary costs for mistakes, operate under extremely limited host attention, handle rapidly interleaved topics, and survive long sessions that may be interrupted across devices. These constraints push the agent beyond the simple personal‑assistant scenario and demand a robust engineering skeleton.

Harness Core Components (E, T, C, S, L, V)

Execution Loop : drives the "think‑act‑observe" cycle and guards against uncontrolled loops.

Tool Registry : defines what tools can do, their limits, and how errors are reported.

Context Manager : maintains context quality and size, preventing token explosion and attention drift.

State Store : persists session state for precise recovery after interruptions.

Lifecycle Hooks : inject strong rules at key points without altering the model’s reasoning loop.

Evaluation Interface : provides observable metrics for continuous assessment.

Layered Architecture: Framework vs. Business Skills

The framework layer supplies stable engineering capabilities—execution loop, context governance, safety, persistence, and observability—while business developers only need to implement Skills that declare capabilities, risk levels, and validation schemas. This clean separation enables rapid feature iteration without compromising safety.

Five‑Layer Security Defense

Prompt‑boundary hard‑coding of allowed actions.

Schema‑level strong type constraints and idempotent keys.

Risk‑based approval gates (auto, soft‑gate, hard‑gate) with tiered reviewer responsibilities.

Tool‑execution verification that applies automatic retries or aborts based on structured error codes.

Comprehensive audit logging of code, output, user, and session identifiers for traceability.

Context Engineering

To keep the model’s context fresh and focused, a three‑step compression strategy is applied when token usage exceeds thresholds: compress historical tool calls, summarize past dialogue rounds, and compress the current round. Long conversations are segmented by session topics (pre‑live, on‑live, post‑live) and stored as structured summaries.

A reducer‑style state update replaces naïve appending of full tool‑call JSONs. Each round injects the latest serialized state via a system hint, ensuring the model sees a concise, deterministic snapshot rather than a sprawling history.

Tool Invocation Enhancements

Ability Boundary Declaration : each Skill registers its permissible actions; the framework validates requests before execution.

Schema Constraints + Idempotency : JSON Schema validates parameters; idempotent keys prevent duplicate side‑effects.

Structured Error Codes & Auto‑Recovery : error categories (3xxx business, 4xxx parameter, 5xxx system, 9xxx unrecoverable) trigger specific fallback strategies such as retry, alternative plan, or user notification.

Lifecycle Hook Integration

PreReasoning : inject latest state and relevant memory into the system prompt.

PreToolCall : enforce ability checks, idempotency, and risk‑based approval.

PostToolCall : validate tool results and update the reducer state.

PostReasoning : perform hallucination detection by cross‑checking model output with real data.

OnSessionEnd / LiveEnd : write session traces back to the memory store.

Sandbox Execution Protection

All code‑type tool executions run inside a non‑privileged container with read‑only root filesystem, CPU limits (≤50%), process caps (≤64), network deny‑all with minimal allow‑list, and strict syscall whitelisting. Output size is capped at 64 KB, and any deviation is truncated to keep the context clean.

PlanEngine: DAG‑Based Global Planning

Complex multi‑step host commands are decomposed into a directed‑acyclic graph (DAG) rather than a linear ReAct loop. The engine provides:

Three‑level checkpointing for resumable long‑running tasks.

Hierarchical trace IDs for end‑to‑end observability.

Parallel execution of independent sub‑tasks to improve efficiency.

Global optimization that reduces tool‑call redundancy and iteration count, achieving higher success rates (PlanEngine 0.847 vs. ReAct 0.737 in internal benchmarks).

Memory System Tailored for Live‑Streaming

The agent’s long‑term memory is split into three layers:

L1 – Session Memory : host‑declared preferences, constraints, and feedback.

L2 – Fact Memory : immutable data such as SKU details, pricing, exposure metrics.

L3 – Behavior Memory : aggregated host actions, audience patterns, and historical performance.

A reconciliation mechanism compares L1 statements with L3 observations. Consistent signals increase confidence; repeated contradictions trigger a prompt to the host for confirmation before overwriting preferences.

Decision‑Trace logs record every suggestion, the host’s response, and downstream lift. These logs feed a trust‑score that quantifies the host’s confidence in the agent. Trust updates follow a simple rule set (e.g., +0.05 for accepted suggestions with positive lift, –0.10 for accepted but harmful suggestions). The trust score drives output style:

Trust ≥ 0.7 → full recommendation with alternatives and confidence.

0.4 – 0.7 → evidence‑only, weakly worded suggestions.

Trust < 0.4 → data presentation without prescriptive advice.

Memory forgetting uses multi‑factor weighted decay (scene relevance, freshness tier, time, and credibility) rather than pure time‑based decay, ensuring that high‑confidence facts persist while stale or low‑trust items fade.

Evaluation Framework

Two complementary tracks measure quality:

Offline : curated benchmark datasets for pre‑live, live, and post‑live scenarios, including adversarial edge cases.

Online : real‑time dashboards track operation success rate, approval pass rate, host‑intervention frequency, end‑to‑end latency, and post‑session host satisfaction scores (1‑5).

Trace visualisation (e.g., Langfuse) and audit logs enable root‑cause analysis, model improvement data collection, and dispute resolution.

Conclusion

The Harness framework—six‑tuple architecture, layered responsibilities, deep security, DAG planning, and a purpose‑built memory system—transforms an uncertain LLM capability into a controllable, observable, and evolvable live‑streaming host agent. While model performance will continue to improve, the engineering skeleton described here is the decisive barrier that turns a demo into a production‑ready product.