Uncovering AI Product Design Challenges: Insights from OpenAI and Anthropic CPOs

Embracing Uncertainty

From determinism to uncertainty: AI product roadmaps must be continuously re‑evaluated because model capabilities evolve rapidly and future performance is hard to predict.

Randomness and model behavior: Large‑language models generate the next token probabilistically; identical prompts can produce different outputs. Temperature adjusts confidence but does not guarantee repeatable results.

The “lumpy nature” of AI: Models excel on some tasks and struggle on others, creating an uneven performance distribution that requires thorough testing to identify viable industry scenarios.

Planning and Iteration Amidst Uncertainty

“Squinting through the mist”: Forecasting specific AI capabilities for a given domain remains uncertain despite scaling‑law trends; practical experimentation is needed to uncover real‑world performance.

Iterative co‑design: Close collaboration between product, research, and fine‑tuning teams (co‑design, co‑research, co‑fine‑tuning) is essential, though siloed KPIs can hinder cross‑team interest.

Experimental culture: Build, validate, and close the commercial loop in real scenarios; avoid chasing a perfect initial product.

Adapting Product Management for the AI Era

AI sense: Product managers need a solid grasp of AI fundamentals and intuition about model behavior, not merely translation of business requirements.

Evals as a core skill: Define success metrics, understand evaluation techniques, and iterate based on data; robust evals are critical because deploying a black‑box model without them can be disastrous.

Data‑driven intuition: Repeated hands‑on analysis of model outputs cultivates tacit expertise that differentiates high‑performing PMs.

Continuous learning with AI: Use AI itself to explore evaluation methods and sharpen the ability to measure model effectiveness.

Human‑AI Collaboration Design

Human‑led, machine‑assisted: Design assumes humans lead while models assist, with graceful‑failure mechanisms and Human‑in‑the‑Loop safeguards.

Collaboration patterns: Beyond explicit feedback (likes, clicks), allow users to correct errors, provide additional context, and discover effective collaboration models for specific domains.

Designing for Non‑Deterministic UI

Model output variability breaks the fixed‑input‑fixed‑output UI assumption. A hybrid GUI + CUI approach can handle deterministic high‑frequency flows with GUI while using CUI for long‑tail, open‑ended interactions.

Building Trust Through Transparency

Admitting bias and limits: Openly disclose model biases and potential failure modes.

Explainability: Although foundational model interpretability is limited, product interfaces can surface reasoning traces to improve user trust.

Feedback mechanisms: Explicit and implicit feedback loops (e.g., Google Learn About’s Interactive Guides) are vital for continuous improvement.

Empowering Power Users

Early adopters: Identify enthusiastic internal “super users” to gather rapid feedback and act as evangelists.

Iterate for power users: Early‑stage feedback drives product vitality; solutions must address real problems rather than showcase flashy technology.

Future Outlook

Proactivity and asynchrony: Future AI agents will anticipate user needs (proactive) and handle tasks that tolerate delayed responses (asynchronous), such as longer reasoning phases.

Multimodal interaction: Shift from text‑only to real‑time voice and other modalities, exemplified by OpenAI’s recent real‑time voice API.

Personalization: As base models become more capable, differentiated personalization will become a key competitive factor.

References

Lenny’s Podcast: A Conversation with OpenAI’s CPO Kevin Weil, Anthropic’s CPO Mike Krieger, and Sarah Gu.

Code example

一点思考(1) - "Generative AI’s Act o1" from Sequoia
一点思考(2) - "Collapsing the Talent Stack"
一点思考(3) - 关于大模型服务领域落地的一些思考 (2024 NJSD)
一点思考(4) - 从State of AI Report 2024看大模型行业有多卷