How to Quantify User Experience: The FLOE Usability Measurement Framework

We often hear generic solutions like “improve user experience to boost satisfaction,” but in practice it’s hard to prove value because experience is subjective and complex.

When optimizing or redesigning a product, it’s urgent to objectively measure the impact. What should be measured and how?

First, the team must align on a shared understanding of “experience” to enable standardized metrics.

Overall Approach: How to Improve User Experience (Satisfaction)?

Use tools such as service blueprints and user journey maps to systematically analyze the scope of influence for the department and stakeholders.

For example, in JD’s POP merchant backend, the overall perception of experience spans many touchpoints across departments. Some improvements require breaking down silos and cross‑department collaboration, while others are within a single team’s control.

Within the actionable scope, decompose factors affecting experience, categorize them (fully controllable, solvable by certain roles, primary vs secondary contradictions), and manage them accordingly.

Establish a standardized evaluation system to guide work direction and quantify value, providing an objective view to complement satisfaction surveys.

We identified improving product usability as the most critical and feasible lever for boosting satisfaction.

FLOE Usability Measurement Framework

Metric Model and Scale

The usability metric system focuses on four core dimensions: functionality, learnability, operability, and efficiency, comprising 19 key indicators that are interrelated.

“Functionality” assesses product usefulness and perceived value; the other three dimensions evaluate usability performance. “Learnability” examines structure, layout, and information organization; “Operability” checks whether users can correctly perform tasks; “Efficiency” measures task completion speed and effort.

The scale consists of up to 19 items scored by users after usability testing, yielding an overall usability score.

Key Features

Advances beyond SUS, WAMMI, USE by detailing 19 indicators across four dimensions, providing concrete improvement directions.

Evaluates the product’s ability to support complete user tasks, adding a “task continuity” metric.

Emphasizes linking user tasks with observations, reducing later research costs.

Usability Measurement Process

Define evaluation plan. Choose heuristic evaluation or usability testing, prepare task cards, data, and equipment.

Recruit participants. Recruit real users matching target criteria; typically 3‑5 experts and up to 20 users.

Conduct testing. Users perform tasks while thinking aloud; observers record issues. Remote testing can use screen recordings and scoring.

Calculate results. Compute usability scores, combine scale ratings, expert feedback, and observations into an issue list.

Hold review meeting. Stakeholders discuss issues, prioritize optimizations, and produce a usability measurement report.

Execute. Development implements improvements based on report priorities; acceptance criteria consider impact, frequency, difficulty, and iteration plan.

Validate. Archive test materials for future reference.

Project Practice Example

The tool helps identify problem products, diagnose issues, and track iterations. It can also serve as a consensus usability metric during prototype testing, structuring observations to support product decisions.

We currently practice two forms: 1) collaborate with the testing department to conduct regular experience reviews and drive product optimization; 2) work with target products to perform diagnostic measurements and produce usability reports for optimization planning.