From Mercury Missions to Modern CI: Mastering Software Testing in Startups

1. Historical Context and Test Classification

Software testing can be traced to the US Mercury space program, where safety and political pressures required systematic verification. Testing evolution is described in five stages: debugging, basic testing, exploratory testing, V‑&‑V (V‑model), and the current phase that emphasizes exploratory, agile, TDD/BDD, and continuous integration. Traditional classifications are complex; the test‑pyramid model (unit, service, UI) simplifies them. Google’s taxonomy categorises tests by size (small, medium, large) and aligns with the pyramid proportions (≈70% unit, 20% integration, 10% system).

2. Automation Tools

Automation tools have progressed from linear script recorders to structured, data‑driven and keyword‑driven frameworks. Two main categories exist:

UI‑based tools simulate user actions (mouse clicks, keystrokes, etc.).

API‑based tools invoke services directly, bypassing the UI.

3. Conditions for Automation

Automation is valuable when it can replace repetitive, error‑prone tasks. Benefits include reduced manual effort and consistent execution; drawbacks are high upfront cost, maintenance overhead, and limited ability to discover new defects. Automation is most effective for long‑lived, stable products with frequent iterations. It should complement, not replace, manual exploratory testing.

4. Agile Testing Principles

Agile testing treats testing as a continuous, team‑wide activity. Key practices are:

Shift‑left: involve testers in requirement reviews and test design early.

Shift‑right: monitor production behavior and feed back issues.

Perform full‑regression testing each iteration to build a robust test suite.

Adapt processes to the team’s context rather than imposing a one‑size‑fits‑all model.

5. DevOps and Continuous Testing

In a DevOps pipeline, testing acts as a gate between development, QA, and operations. Continuous Testing (CT) works alongside Continuous Integration (CI) and Continuous Deployment (CD) to provide rapid feedback and quality assurance throughout the delivery flow.

6. GitLab Test Integration

Integrate automated test tools into GitLab CI/CD so that every commit triggers comprehensive testing. Merge‑request pages should display unified test reports linked to the specific code changes, enabling reviewers to approve only when tests pass. This creates a “test gate” that enforces quality before code merges.

7. Practical Setup for Start‑ups

Prerequisites:

Version‑control system (e.g., Git).

CI/CD pipeline for automated execution.

Static analysis tools to catch basic issues when dedicated QA resources are unavailable.

Testing order recommendation (based on the test pyramid and startup constraints):

Interface testing – low entry barrier using visual tools; automate via containerised scripts for stability and early feedback.

Performance testing – after interface validation; select load, stress, soak, or spike tests based on clear non‑functional objectives (refer to Microsoft’s performance testing workflow).

Unit testing – developers write tests for critical modules; focus on pass rate, defect trend, and meaningful coverage rather than arbitrary high coverage percentages.

UI testing – automate repetitive UI flows (e.g., Selenium headless mode) while keeping visual validation manual when cost outweighs benefit. Use GUI mode with dedicated hardware for complex interactions.

8. Q&A Highlights

Team staffing : Start with developers handling testing; add part‑time testers as the product stabilises; expand the dedicated testing team once the organisation exceeds ten engineers.

Frequent logic changes : Defer automation until the logic stabilises; rely on manual testing for rapid iteration.

Performance testing timing : Introduce early for high‑traffic consumer products; run performance tests every two weeks in agile cycles or monthly in waterfall projects.

Code example

阅读原文，跳转我的仓库地址