Embracing Uncertainty: Redesigning AI Product Requirements

Why Traditional PRDs Fail for LLM‑driven Chatbots

Conventional product requirement documents assume deterministic behavior: a button click always produces the same UI, missing data always triggers a fixed error. Large language models (LLMs) generate probabilistic outputs, so a static flowchart cannot capture all possible user utterances. When a deterministic PRD is applied, the model either repeats scripted replies or drifts into unrelated topics (e.g., “quantum mechanics”), breaking the intended conversational path.

Shift to Value‑Based Constraints

Instead of enumerating every edge case, define high‑level value constraints that bound the model’s behavior. Example:

Value Constraint: The system should act as a cautious, professional financial advisor. For any request that touches a legal gray area, it must remain neutral, issue a risk warning, and prioritize the user’s asset safety.

This replaces explicit logical rules (e.g., “if user asks about illegal loans, reply ‘I can’t answer’”) with a boundary that guides the model across a wide range of inputs.

New PRD Structure for Uncertain Outputs

Evaluation Set : A curated collection of representative dialogues covering typical, edge, and adversarial scenarios. Each entry includes the user prompt, expected value‑aligned response, and a “golden case” label.

Dynamic Benchmarks : Continuously updated performance targets (e.g., acceptable fuzziness score, recall of risk‑warning intents) that reflect evolving model capabilities and data.

Prompt Requirements : Explicitly state where the model may be creative (e.g., storytelling) and where it must remain silent or give a safe fallback (e.g., medical advice).

Hallucination Handling : Define a correction workflow—either a gentle self‑correction phrase or integration of Retrieval‑Augmented Generation (RAG) to verify factual claims.

Unacceptable Behavior List : A blacklist of actions the model must never perform (e.g., providing disallowed financial advice, revealing personal data).

Metric for Probabilistic Evaluation: Fuzziness Assessment

Traditional pass/fail testing is replaced by a fuzziness metric that measures how closely the model’s output stays within the defined value constraints. Typical calculation:

Fuzziness = 1 - (Number of constraint violations / Total evaluated turns)

A lower fuzziness score indicates tighter adherence to the value boundary.

Data‑Centric Requirements

Because the PRD now serves both developers and data teams, it must specify:

Golden Cases : High‑quality examples that illustrate the ideal response style and content.

Negative Feedback : Annotated instances where the model deviated from constraints, used for fine‑tuning or reinforcement learning.

These sample‑based definitions are more efficient than exhaustive textual logic.

Practical Workflow

Draft value‑based constraints for each product domain.

Build an Evaluation Set with annotated golden and negative cases.

Define Prompt Requirements and the Hallucination Handling strategy (self‑correction vs. RAG).

Run automated fuzziness assessment on the Evaluation Set.

Iterate constraints and data annotations until the fuzziness metric meets the Dynamic Benchmark.

This approach transforms the PRD from a rigid rulebook into a living agreement that guides LLM behavior while preserving safety and product intent.