Baidu Product Evaluation Framework and Common Assessment Methods

With increasing competition in the Internet industry, product quality, service reliability, and user experience have become decisive factors for user choice, prompting QA teams to focus more on overall service quality and user‑centric product evaluation.

Product evaluation combines subjective research (surveys, expert reviews, user observation) with objective data (metrics, big‑data behavior analysis, performance stability) to verify both functional availability and usability in real user scenarios, ensuring findings are authoritative and recognized by all product roles.

1. QA Evaluation Scope and Value

Core function layer – assesses resources, data, content quality, performance, stability, and policy effectiveness.

Product layer – checks whether product effects meet user expectations and identifies bad cases.

User & client layer – measures user and client experience through feedback and sentiment analysis.

Market & industry layer – evaluates product success against market goals via competitive and metric analysis.

2. Evaluation Methods and Tools

Basic tool layer – shared services and libraries used across product lines.

Method service layer – objective methods (data statistics, competitor comparison, feedback analysis) and subjective methods (internal testing, questionnaires).

Application layer – concrete implementations such as automated metric monitoring, online bad‑case mining, and competitor analysis.

Support layer – common platforms and tools provided by the quality department.

3. Common Evaluation Methods

3.1 Indicator Statistics & Data Analysis – Collect and analyze data to build a comprehensive indicator view, turning metrics into reusable cases that can be applied throughout the product lifecycle.

3.2 Strategy Effect Evaluation – Small‑traffic experiments (ABTesting, Interleaving) compare different versions to assess impact on user experience.

ABTesting A/B testing presents two variants to separate user groups, records behavior, and determines the better solution; advantages include simplicity and reliable conclusions, while drawbacks are traffic consumption and potential user‑group bias.

Interleaving Merges A and B results and serves the combined list to users, then observes behavior; includes Balance and Team‑Draft interleaving.

3.3 User Feedback & Sentiment Analysis – Collects generic feedback (forums, social media), precise feedback (internal platforms), and implicit feedback (clicks, query changes) to close the feedback loop.

3.4 User Behavior Analysis – Analyzes implicit feedback through path analysis and bad‑case mining.

3.4.1 User Behavior Path Analysis – Generates user visit sequences and adjacency matrices to visualize page transition ratios and identify abnormal flows.

11:20 用户1 home 11:21 用户2 home 11:22 用户1 abroad 11:23 用户2 user_order

3.4.2 Bad‑case Mining from Behavior Data – Combines query, resource ID, and position with click/turn‑page signals to rank candidate bad cases for manual review.

3.5 Big‑Data User Portrait Analysis – Leverages Baidu’s user‑portrait platform to obtain personal and group attributes across 7 dimensions and 100k+ tags, supporting product optimization and competitive analysis.

3.6 Competitor Analysis – Evaluates function, performance, effect, and market metrics against rivals using automated monitoring and metric comparison for web and app products.

3.7 User Experience Evaluation – Uses surveys, expert reviews, and models such as the 5E or honeycomb framework to assess coverage, ease of use, interaction, and visual design, supported by internal testing platforms.