Three Key Questions for Effective Software Testing: Effectiveness, Efficiency, and Value

Recently I have been pondering what the three key questions for testing should be. I have some initial hypotheses and solutions, and I write them down to invite feedback and discussion.

Problem 1: Is the test truly effective? — Test Effectiveness

The first key question is whether my tests are truly effective. It may look like a pseudo‑question, but in reality we should ask whether we can guarantee that all tests we run are effective. Experience shows that the proportion of effective tests is often low, and many testers never consider whether large‑scale tests are truly effective.

How to evaluate test effectiveness? Consider test strategy, test feedback, and testability.

1.1 Test Strategy

First, assess the test strategy using the test‑pyramid model:

(Test Pyramid)

Total layers: which tests are performed at each layer.

Proportion of each layer: focus of testing.

Goal of each layer: different tests serve different objectives.

Coverage requirements per layer: what issues each test should cover and the required coverage level.

Then evaluate the effectiveness of test cases and test suites:

Every manual or automated test case must be effective.

Different test suites should genuinely serve their respective testing goals.

Eliminate duplicate tests or overlapping coverage across layers to avoid waste.

Ensure relatively complete coverage that emphasizes business‑scenario coverage rather than raw coverage numbers.

Establish a test‑drip mechanism: if a defect is found in an upper layer, assess whether it escaped lower layers and add tests there if needed.

1.2 Test Feedback

To make testing effectively feedback code quality, an appropriate amount of testing should be performed at the right time, provide effective feedback, and act as a gate to prevent low‑quality code from flowing downstream.

Appropriate amount of testing: different tests should execute a suitable number of cases—less is more as long as coverage is maintained.

Right timing: tests can be scheduled periodically, run on‑demand, or continuously in a pipeline.

Effective feedback: realistic expectations, correct assertions, and true reflection of software behavior.

Gate function: when a test fails, it should immediately block further progression to avoid low‑quality code propagation.

1.3 Testability

Software testability is the degree to which the software under test can support testing in a given environment. Factors include controllability, observability, isolation, readability, and automation level. High testability is a prerequisite for effective testing.

Frontend testability: support for UI standards, front‑end code conventions, etc.

Backend testability: high cohesion, low coupling, test interfaces, controllable and observable execution steps.

Comprehensive logging: provides observability and traceability for rapid issue location.

(Log system that facilitates observation)

Problem 2: Can tests be executed efficiently? — Test Efficiency

When test effectiveness is ensured, we must also focus on how efficiently tests are executed, because testing occurs within limited time windows.

2.1 Test Environment Support

The usage of test environments can be roughly divided into the following steps:

Preparation of test resources → Test environment deployment → Test service deployment → Environment verification → Environment usage → Environment teardown

In practice, test environments often become a major bottleneck. Different testing goals require different environments, data, and integration setups, so we expect test environments to be dedicated, isolated, and free from interference between different tests, testers, or data.

(Complaints about test environments)

Testers expect environment support to meet several conditions: low deployment and maintenance cost, on‑demand scaling, disposable usage, and isolation of environment and data across different tests, allowing them to focus on business testing.

(Desired new environment that does not exist)

2.2 Test Infrastructure

Achieving high test efficiency requires continuous integration. The ideal test strategy consists of a small amount of manual exploratory testing, a large amount of automated regression testing, and requirement‑driven special testing, with the automated regression tests integrated into the CI pipeline.

Evaluation checklist:

Presence of a large number of effective automated tests.

Automated tests integrated into the CI pipeline.

Core module unit tests or core business flow regression tests triggered on each code commit as independent pipeline steps, providing effective feedback on code quality.

Other business regression tests executed regularly and especially at critical milestones.

Visualization reports or monitoring mechanisms for test execution to ensure timely issue handling.

Beyond CI, effective testing also depends on rapid test‑suite preparation, data generation for different test goals, and tool or platform support.

2.3 Test Execution Efficiency

Regardless of how complete the environment and infrastructure are, testers must continuously monitor and improve test efficiency:

Manual exploration: does it help discover defects within limited time, and should its findings be added to regular test cases?

Automated testing: reasonable execution cycles, stable duration, and a maintained pass rate.

Problem 3: Where does test value manifest? — Test Value

3.1 Built‑in Quality

Previously testing’s responsibility was to find software defects; now we talk about built‑in quality, which means discovering and correcting process defects. Any practice that can lower software quality becomes a target for improvement.

In the built‑in quality process, the greatest value a tester provides is helping the whole team develop a quality mindset, shifting from “testing and quality is QA’s job” to “testing and quality is everyone’s job.” This mindset shift is difficult but, once achieved, quickly aggregates benefits across all work.

(Full‑process built‑in quality)

3.2 Exposing Risk

Testing also has the important duty of fully exposing risk, which includes quality risk, delivery risk, and production‑environment risk. Practices such as defect modeling, defect data analysis, root‑cause analysis, and defect prevention enable testers to understand quality risk and predict potential issues before release, reducing production‑environment risk.

(Tester’s responsibility: identify and expose risk)

3.3 Guarding the Quality Gate

Testers must defend the quality gate: if a test fails, it must stay failed, and clear quality conclusions should be given to avoid ambiguous outcomes. Example conclusions include:

“The feature has been fully tested and regressed; it passes and can be released. Post‑release monitoring is required.”

“Due to tight schedule, the feature only meets acceptance criteria and lacks sufficient regression; we recommend postponing the release.”

“If release is unavoidable, the following quality risks exist… recommended mitigations and rapid recovery steps are provided, with post‑release monitoring to capture issues promptly.”

If the gate is compromised, the whole team shares the risk and must work together to minimize loss.

Write at the End

I continue to reflect on this topic and wonder why testers often lack a sense of value, perhaps because many of their activities seem meaningless. I hope this thinking process helps me find ways to give testing real meaning.