Testing and Evaluation Practices for Baidu Doctor Platform

Baidu Doctor is dedicated to building a professional doctor‑patient matching platform in China, enabling users to quickly book nearby doctors, reduce appointment costs, and achieve reasonable allocation of medical resources. It creates a closed‑loop online service chain of finding, booking, and evaluating doctors to improve matching efficiency and experience.

In this closed‑loop service, Baidu Doctor continuously expands product breadth and depth by integrating resources such as Yihu.com and Haodf, enriching hospital and doctor data. It offers multi‑dimensional vertical search (hospital, doctor, disease, department) with guidance for precise queries, and refines services like timed appointments and online payment. Since launching the mobile web version in Oct 2014, the APP in Jan 2015, and topping Apple’s medical APP chart by Dec 2015, the testing team has consistently ensured quality through user experience data analysis and retrieval side testing.

1. User Experience Data Analysis

As an O2O product, online services ultimately affect offline experiences; perfect online quality does not guarantee flawless offline service. Therefore, Baidu Doctor faces unique quality assurance challenges compared to traditional internet services.

1. Fully Trust Source Data

Third‑party source data (e.g., doctor information) is trusted for validity, but updates may cause data expiration and Badcase issues. Real‑time online data retrieval and monitoring strategies are employed to detect such Badcases.

Figure 1: Source Data Monitoring Scheme

2. Collection and Analysis of Real Online User Experience Data

Online monitoring is conducted to ensure user experience, with the functional logic shown below.

Figure 2: Baidu Doctor Monitoring Framework

From a business perspective, online monitoring includes:

a) Basic page element monitoring – images, links, text, data, etc.

b) Core data interface monitoring – validation of HTTP interface format, size, and content.

c) Third‑party API monitoring – early detection of external service issues.

d) Machine hardware availability monitoring – ensuring server hardware health.

Figure 3: Interface Automated Monitoring Scheme

The automated monitoring workflow consists of:

1) Automatic online log collection.

2) Log analysis to automatically generate monitoring cases.

3) Execution of monitoring with alerting.

This approach automatically creates monitoring cases from interface responses, greatly reducing manual maintenance costs.

Log monitoring also collects all user service request logs, analyzes product defects and user behavior, and guides product improvement.

Figure 4: Log Monitoring Scheme

2. Retrieval Side Testing and Evaluation

The retrieval side powers Baidu Doctor’s web and APP front‑ends, providing search for doctors, hospitals, diseases, departments, and symptoms, as well as suggestion and filtering services. Quality assurance covers both offline and online testing, with a comprehensive evaluation plan illustrated below.

Figure 5: Retrieval Side Quality Assurance Mind Map

1. Product Line Specificity

Medical data professionalism requires covering specialized queries, handling user search habits, and addressing ambiguous entity recognition. Privacy concerns limit direct user surveys, so log‑based analysis is used to model user jumps and churn.

Additional satisfaction indicators include the proportion of top‑tier hospitals and chief physicians in results, and the impact of hospital distance ranking on user satisfaction.

Resource quantity and quality (regional coverage, hospital level, doctor level) directly affect user experience, prompting comparative evaluations with competitors.

2. Retrieval Side Testing and Evaluation

Key work includes:

2.1 Self‑Testing with QA+RD Collaboration

Before delivery, RD conducts self‑testing using QA‑provided libraries, ensuring coverage. The xts backend automation framework, its libraries, test cases, and source code are managed together.

2.2 Test Automation

From code checkout to environment deployment, daily builds, and release, QA configures Jenkins jobs for one‑click automated execution.

2.3 Retrieval Evaluation

Monthly evaluation reports compare pre‑ and post‑optimization retrieval performance. Evaluation includes queries from Baidu’s medical search and real logs, comparing Baidu Doctor with competitors and across versions.

Evaluation metrics:

Accuracy – correctness of retrieved hospitals, doctors, departments, diseases, symptoms.

Recall – completeness of retrieved results.

Authority – ranking and count of top‑tier hospitals and chief physicians in results.

Scoring combines weighted sub‑metrics to produce overall scores.

2.4 Retrieval Badcase Mining

Identified issues are categorized, rules are derived, and targeted Badcase mining is performed weekly, generating reports covering doctors, hospitals, diseases, and departments.

Figure 6: Badcase Automatic Localization Process

Hospital Badcase localization distinguishes between missing resources and strategy issues, using entity recognition and resource lookup, followed by strategy module analysis.

Figure 7: Hospital Badcase Automatic Localization Flow

2.5 User Real Search Habit Analysis

Analysis includes user jump and churn across search pages and tolerance for repeated searches after unsatisfactory results. Raw logs are clustered by user ID, ordered chronologically, mapped to sequences, stored in matrices, and statistical ratios are visualized.

Figure 8: User Search Habit Analysis Process Diagram

Follow the Baidu Quality Department subscription account for more insights. Reply with keywords such as “evaluation”, “CI”, or “mobile testing” to receive related articles and tool reviews.