Bridging the Gap Between Academic Research and Engineering Practice in Software Testing

Author Introduction

Yang Xiaohui – author of “The Road to Enhancing Software Testing Value”, former chief testing expert at Huawei Technologies Co., Ltd. Software Company. Joined Huawei in 1999, led product testing, testing‑process transformation, and test‑engineer career development. Since 2007 she has overseen testing architecture design, implementation and application, helping products continuously accumulate testing capabilities and improve R&D efficiency and quality.

Engineering application and academic research have huge differences in understanding and interest in testing technology.

One year ago, Professor Zhu Shaomin published a series of “Software Testing Technology Application Status Survey Report” on his WeChat public account “Software Quality Report”, revealing survey results of university, research‑institute and software‑company practitioners regarding their understanding, application, interest and hot topics in testing technology.

Unsurprisingly, there is a huge gap between software companies and universities:

◆ Testing technologies that have matured for over 20 years are relatively common in both companies and schools.

◆ Model‑Based Testing (MBT) that has been proposed and applied for over 10 years still remains mostly in universities.

◆ More recent search‑based testing methods are almost still in universities, with very few entering engineering applications.

It can be seen that the hot topics in software testing differ between engineering application and academic research; academia focuses more on algorithms and is full of confidence in AI, while engineering application emphasizes standardization, componentization of software development, as well as agile, DevOps and other engineering capability improvements.

Testing is not an exception; in fact, almost all technology fields have a gap between application and research, which is natural.

▌ Academic research needs breakthroughs in specific directions:

▌ Engineering application needs to use multiple dimensions of mature, efficient knowledge to solve problems:

This does not mean engineering application and academic research will diverge; on the contrary, engineering application follows academic research, continuously turning research results into convenient tools that solve enterprise problems. Initially a few pioneers implement this connection, then more engineers adopt the technologies, and eventually new technologies become embedded in tools and methods, becoming easy to use.

Those sharp engineers, or those who can quickly master new tools, appear “smart” in teams because when a particular problem stands out during product development, they can proactively lead test‑driven improvement projects. Such people usually track industry information long‑term, enabling them to find the right tools when problems arise.

To give “smart” advice at such times, the key is whether testing has sufficient “breadth” of knowledge and whether one follows industry news, mature methods and tools, and accumulates product‑related problem analysis. Rich, structured testing information can give development teams confidence to solve problems, supporting evaluation and advancement of solutions based on testing confirmation (i.e., test‑driven improvement).

General information collection can be performed from three angles:

◆ Look at the industry – information about the whole software industry and testing field.

◆ Look at peers – information about similar products, domains, or other products within the same company.

◆ Look at oneself – methods, progress, problems and impacts of one’s own product.

From these angles, the information first helps form a systematic view. It is recommended to start with a “testing treasure map” (software testing at a glance) or a “software testing panorama” (several versions can be found online; choose a recent one). Using this map as a guide, you can obtain a framework understanding of testing technologies. The map contains a huge amount of information; mastering every field in the short term is impossible, so first understand what each area is and what problems it can solve.

From these angles you can also learn about current hot technologies and novel problem‑solving ideas. International examples include the STAR series conferences (STAREAST, STARWEST, EUROSTAR) and technical conferences organized by top companies such as Microsoft and IBM. Domestic examples include TiD, CSTQB summit, MSUP TOP100 cases, Alibaba testing carnival, etc. Conference topics reveal which issues, tools, technologies and viewpoints are currently popular in testing engineering. Of course, hot technologies should be viewed critically; they can be beneficial but may be immature and require you to fill gaps.

However, reliability and security testing technologies are shared less through these channels; more often you need to read articles and papers on software safety testing and software reliability testing to obtain related information.

Another important channel is learning from people around you; internal team testing engineers can share regularly. Our team has an internal forum where each person regularly presents their work and showcases core results, achieving knowledge sharing.

In summary, although many testing engineers in companies seem to use “outdated” methods and tools, expanding horizons and inspiring ideas is still needed, which helps personal growth and work efficiency.

Finally, a word: a systematic view of test engineering is very important; solving engineering problems requires knowledge from multiple domains, and a systematic view helps address dimensional issues. Choosing the wrong dimension often leads to inefficient solutions.

Recommended reference materials:

Information from the “Software Testing Technology Application Status Survey Report” can be found on Professor Zhu Shaomin’s WeChat subscription “Software Quality Report”.

The Road to Enhancing Software Testing Value also systematically introduces methods for solving common problems and can be consulted.