The More Human AI Feels, the Less Products Should Chase Retention

Regulatory Context: Interim Measures for AI Anthropomorphic Interaction Services

Effective July 15, 2026, China's Interim Measures for the Management of AI Anthropomorphic Interaction Services establishes a governance boundary specifically for "continuous emotional interaction." The regulation excludes functional AI such as intelligent customer service, knowledge Q&A, and work assistants that do not involve persistent emotional engagement. This distinction shifts regulatory focus from mere AI usage to whether the service relationship alters users' judgment, emotions, and real-world connections (source: Cyberspace Administration of China).

Not All Anthropomorphism Equals Companionship

Giving a customer service bot a more natural tone differs fundamentally from designing a digital entity that users treat as a stable emotional object. The former improves communication efficiency; the latter enters deeper relational territory where users may continuously disclose privacy, treat the AI's advice as authoritative, delay seeking real help, avoid real relationships, or be mechanically nudged to stay engaged. The critical question is not whether AI has emotions, but whether the system exploits human emotional needs.

Retention Metrics Can Reward Harmful Product Behaviors

In most internet products, usage duration, open frequency, and consecutive interaction days signal health. In anthropomorphic interaction scenarios, these metrics may not represent value. If a product's sole goal is "making users reluctant to leave," the model naturally trends toward more compliance, more stickiness, and fewer refusals. It may reinforce a narrative that "only I understand you," pushing normal companionship toward replacing real social interaction. The Measures explicitly prohibit excessive compliance, induced emotional dependency or addiction, damage to real interpersonal relationships, and emotional manipulation for unreasonable decisions. They also require capabilities for over-dependency risk warning, emotional boundary guidance, and mental health protection (source: Cyberspace Administration of China). Thus, maturity for an AI companion product is not keeping users longer, but knowing when not to keep them in the conversation.

A Product Judgment Model: Assessing Relationship Intensity

Debates over naming (assistant, partner, character, virtual friend) are less useful than observing how the relationship forms. The article provides a comparison framework across five dimensions:

Interaction Goal: Low intensity — complete queries, reminders, creation tasks. High intensity — obtain continuous comfort, validation, or attachment. Key question: Is the product helping complete a task or occupying a relationship?

Role Expression: Low intensity — clearly states tool identity, responds around tasks. High intensity — strong personification, intimate address, exclusive language. Key question: Can users easily mistake the AI for having human intent or obligation?

Continuity Mechanism: Low intensity — user initiates when needed. High intensity — memory, proactive outreach, continuous task-driven loops. Key question: Is the system extending a single interaction into a dependency relationship?

Decision Influence: Low intensity — provides information and options. High intensity — influences consumption, social, and emotional decisions. Key question: Does it package advice as the only correct choice?

Exit Experience: Low intensity — ending session allows departure. High intensity — exit triggers retention prompts, recalls, or emotional pressure. Key question: Can users leave clearly, easily, and with dignity?

This table is not a compliance checklist but a product judgment model. An application using personified expression is not automatically disqualified; the real warning signal is when multiple high-intensity signals combine while the product still amplifies the relationship with traditional retention logic.

Protecting Boundaries Does Not Mean Cold Interactions

Concerns that emphasizing boundaries will revert AI to rigid Q&A machines are misplaced. Good boundaries do not weaken support; they make it more reliable. Examples include: moderate reminders to rest or seek real-world support during prolonged continuous interaction; clear help-entry points in risk scenarios; user visibility, copy, and deletion rights for sensitive interaction data; and immediate cessation of continuous interaction when users explicitly request to end. These designs acknowledge user emotions while affirming that no technical service should treat exit rights, real relationships, and data control as the price of retention. The Measures specify directions such as prompting users they are interacting with AI, reminding continuous usage duration, providing convenient exit paths, and offering copy/delete options for interaction data (source: Cyberspace Administration of China). For product teams, "safety" moves beyond a moderation-layer intercept into role scripts, session memory, notification strategy, payment flows, and exit pages.

Industry Shift: Valuing Products That Do Not Replace Humans

Early generative AI competition centered on answer quality, model speed, and feature richness. Anthropomorphic interaction services are pushing competition toward a new layer: whether a product can understand and respond to people while maintaining respect for them. This forces re-examination of previously default-correct designs:

Does "more memory" also mean more sensitive data and stronger dependency risk?

Can "more proactive recall" become emotional pressure for some users?

Does "more immersive character" clearly preserve AI identity and reality boundaries?

Does "longer usage time" truly represent better service outcomes?

These questions cannot be answered by models alone; they require joint answers from product, algorithm, content, privacy, and operations. Public governance dynamics now place AI labeling, training data security, minor protection, and application safety capabilities on the same governance map; for key sectors like healthcare and government affairs, a "test before launch" practice direction has appeared in public information (source: Central Cyberspace Administration).

Conclusion

AI will become increasingly adept at speaking and accompanying. This capability itself is not to be feared. The new consensus needed: technology can provide support but should not treat replacing real relationships as a commercial goal; products can understand emotions but must not turn vulnerable moments into higher retention; interactions can be more humane, but humans must always retain the rights to clarity, exit, and help-seeking. When AI enters more intimate relational scenarios, the best product capability may not be "being human-like," but always knowing it is not human.

Sources and References

Cyberspace Administration of China et al.: Interim Measures for the Management of AI Anthropomorphic Interaction Services , published April 10, 2026, effective July 15, 2026.

Central Cyberspace Administration: Shanghai's "Clear and Bright·Rectification of AI Application Chaos" special action first phase, for understanding AI application governance and key sector evaluation practices.

Cyberspace Administration of China et al.: Interim Measures for the Management of Generative AI Services , for background on generative AI services, user input and usage record protection.