How to Quantify a ‘Good Game’ Early: NetEase’s Six‑Year Evaluation System

Introduction

At GDC 2022 in San Francisco, Lei Huo UX delivered 17 talks, including a session by Jia Sheng and Liu Yongcheng on early evaluation of game products, sharing NetEase’s six‑year experience.

Defining the Early Stage

The "early" phase is defined by three aspects:

Only a game concept or demo exists.

The development team is small, typically fewer than ten core members.

Investment and development time are limited, usually under six months.

Key Metrics for a Good Game

Four critical points determine a good game:

Clear highlights and selling points.

Player retention and activity.

Willingness of players to pay.

Ability to spread organically or be shared.

Except for payment, the other three can be assessed in a 15‑30‑minute early‑stage process.

Evaluation Methods

Three approaches are used in early evaluation:

Six‑dimensional expert assessment : compares the target game with competitors to identify gaps and their causes.

CONCEPT100 concept testing : investigates how key concepts spread among users.

"3X10" experience testing : conducts gameplay tests with three player types to gather market feedback.

Practical Lessons Learned

Avoid adding artificial surprises solely to boost scores if they do not align with core gameplay.

Do not limit testing to a narrow player segment; broader player feedback yields more reliable results.

Refrain from using non‑reproducible high‑cost art assets that limit device compatibility.

Do not over‑invest in niche sub‑markets with limited audience potential.

Avoid testing users under 14 years old due to maturity and bias concerns.

NetEase Recommendations

Focus on validating the idea rather than chasing high scores; use evaluation to guide improvements and inspire new concepts.

Shift decision‑makers’ mindset toward supporting promising projects instead of merely increasing metrics.

Maintain continuous user contact throughout development to incorporate feedback into decision‑making.