Why Data Thinking Is the Key to Evaluating AI Agents for Product Managers

When an AI agent spits out seemingly fluent but unusable answers, many new AI product managers instinctively blame prompts, model size, or the algorithm engineers. The deeper issue is a reversal of causality: they are still applying the old "feature‑first" mindset to a fundamentally different kind of product.

1. Cognitive Misalignment: From Feature Building to Data Refinement

Traditional internet products follow deterministic logic—click A, go to B—so a flowchart and UI details are enough. AI products, however, are probabilistic systems fed by data. An AI‑writing‑tool case illustrates this: despite countless UI tweaks and interaction improvements, user retention stayed flat because the team kept focusing on the "generate" button instead of the underlying data that determines output quality.

The core deliverable of an AI product is not a button but the determinism of the model’s output. Like a chef, the model’s skill depends on the ingredients (data) and how it is trained.

2. Work Reconstruction: From Writing Docs to Defining Ground Truth

In conventional workflows a product manager translates business needs into a PRD. In AI teams, merely writing a document quickly marginalizes the PM because algorithm engineers need concrete ground‑truth definitions, not vague requirements.

For a legal‑consultation AI, the engineer can build a fluent model, but the PM must decide whether a particular answer constitutes a hallucinated legal risk. That decision is the essence of data thinking.

Projects often stall not because of technical difficulty but because the PM cannot quantify what a “perfect answer” looks like, leaving the algorithm without a clear optimization direction.

To address this, the PM should dive into the Bad Case pool, asking:

Is this error common or an outlier?

Does our training data lack this scenario?

What additional data do we need to correct it?

Turning Bad Cases into a data‑flywheel means designing every interaction to collect higher‑quality data for the next iteration.

3. Skill Clarification: From SQL Boy to Data Strategist

Data thinking is not about mastering Python or writing endless SQL queries; it is about developing a sensitivity to bias and building a robust evaluation system.

When a model gives a timid response to "women career advice," a data‑savvy PM recognizes the hidden societal bias in the training set rather than merely blaming the model’s worldview.

Evaluation should be multi‑dimensional, not just accuracy. A useful radar includes:

Safety : Does the output contain hallucinations?

Consistency : Is the tone coherent?

Usefulness : Does it truly solve the user’s problem or just add fluff?

Conclusion

The best AI product manager acts as a "data shepherd" for the algorithm team—knowing which data sources are rich, where risks (wolves) lurk, and whether the model (the flock) is healthy. After classifying 500 weekly Bad Cases, redefining data‑cleaning rules, and iterating the agent, the author finally saw the model speak human‑like language, delivering a sense of achievement that surpasses a simple product launch.

In the AI era, product managers are no longer static builders but gardeners who continuously nurture the soil (data) to keep the model‑plant thriving.