The article describes three automated feedback loops—performance, memory leak detection, and reliability—for a chat assistant Android app, using on-device tracing, heap analysis, and black-box testing to continuously measure, diagnose, and guard against regressions on real devices.
Pinky is an open-source project that enables AI agents to control iOS and Android phones via Wi-Fi, supporting cross-platform desktop integration, secure approval workflows, background operation on Android 13+, and detailed operation logging, demonstrated by hands-on tests posting videos and earning rewards.
360's TEP platform automates LLM evaluation with standardized datasets, dual effect-and-performance benchmarks, built-in and custom evaluators, and reproducible task tracking to replace ad-hoc 'should be fine' decisions with evidence-based iteration cycles.
The article details Taobao Shango's exploration of emotional design for its AI agent, moving beyond selectable tones to a three-layer framework of values, emotional response strategies based on psychology, and quantifiable tone rules, validated through automated testing.
Spawn, a 10-person multiplayer game platform company, uses a fleet of persistent AI agents to handle development tasks from bug reports to deployment, achieving 77,000 changes in 30 days with hourly releases, automated testing every 15 minutes, and AI-assisted decision-making, while humans focus on product experimentation and high-level trade-offs.
Alibaba's 1683 shares its production-grade Agent evaluation framework combining offline golden sets with online log sampling, using Task×Trial×Trace×Outcome decomposition, executable Rubrics with deterministic gates, LLM Judges constrained by evidence citations and rule-based scoring, and a closed loop that routes failures to targeted fixes and regression testing.
Ant Financial's 'one person, three platforms' approach uses LLMs to let a single engineer natively develop for iOS, Android, and HarmonyOS simultaneously, replacing cross-platform frameworks with AI-assisted workflows, shared asset repositories, and automated testing, cutting development hours by 40% and achieving 80%+ AI code adoption.
Researchers from CUHK-Shenzhen and Amphion Technology introduce DuplexAgent-RSI, a full-duplex voice agent system that separates real-time interaction from background task execution via a six-module collaboration layer, and improves its coordination ability through automated testing, failure diagnosis, and verified patches, raising collaboration scores from 54.6% to 85.3% over ten self-improvement rounds.
Anthropic's Claude Code team replaced traditional line-by-line code review with a six-layer automated defense and risk-based human approval, letting AI agents catch bugs while humans only review high-risk changes, cutting review rounds from 8.9% to 1.5% and rollback rates by 40%.
Anthropic's Claude Code team shares their AI-driven code review pipeline that doubled merge rate while cutting rollbacks by 40%, using layered automated checks, risk-based human approvals, and incident-fed evaluation sets to keep reviews convergent and accountable.
The Claude Code team at Anthropic shares their evolved code review process where AI agents write 90% of code, using automated gates, risk-based human approvals, merge queues, feature flags, and incident-driven evals to cut revert rates by 40% while scaling PR volume 10x.
The author used AI to audit the Feat framework, uncovering 9 issues including connection reuse state leaks and security vulnerabilities, all fixed with automated tests, while reflecting on AI's role as a tireless code reviewer and the importance of invisible stability in foundational frameworks.
This article critiques AI-driven self-healing tests for masking real defects and bypassing new business logic, proposing a four-step isolation observation framework—classify failures, attempt temporary recovery, track failure clusters through a state machine, and precipitate reusable knowledge—to make failures explainable, controllable, and accumulative rather than chasing zero failures.
AI in testingautomated testingfailure classification
This article explains the Model Context Protocol (MCP), its architecture and communication methods, then demonstrates how to build and configure a MySQL-MCP server in Python, integrate it with AI tools like Workbuddy, and use browser automation via MCP for automated testing of web applications.
This article details a fully automated AI agent workflow for Laravel using Laravel Boost and OpenCode CLI, where specialized agents autonomously handle building, testing, diagnosing failures, and fixing code in a persistent queue-driven loop until tests pass, with human review only at the end.
The article distinguishes Prompt Engineering (single human-verified interactions) from Loop Engineering (automated iterative loops with testable success conditions), explains their hierarchical relationship, compares use cases, risks, and argues that Loop Engineering builds on Prompt Engineering to automate repetitive, verifiable tasks.
CyberStrikeAI is an AI‑native security testing platform built in Go that integrates over 100 security tools through a multi‑agent orchestration engine, offering role‑driven testing, dynamic task planning, knowledge‑base vector search, MCP protocol integration, and a full vulnerability‑lifecycle workflow for automated red‑team operations.
As AI accelerates code production, the article argues that relying solely on manual code reviews is insufficient and outlines how layered automated regression testing, integrated into CI, provides repeatable evidence to verify real‑world behavior, reduce risk, and build sustainable quality confidence.
The article analyzes how AI-generated code now outpaces traditional testing, exposing coverage blind spots, hallucinated logic, dependency gaps, and latent regressions, and proposes AI‑assisted testing practices to close the validation gap and keep test velocity in step with code creation.
When dozens of services rely on a shared SSO, a simple ping or HTTP 200 does not guarantee users can log in; this article compares common monitoring approaches, explains why a simulated‑login probe using Playwright is chosen, and details the design, implementation, and Linux deployment of the sys‑monitor tool that automatically validates the full login chain and alerts on failures.
The article explains how AI agents can run overnight tests, automatically detect faulty modules, modify production code, and open pull requests, creating a closed-loop evaluation system that shifts testing from post‑hoc error spotting to proactive code iteration, provided three key prerequisites are met.
AI development teams now face a shift from accelerating code generation to tackling the hidden challenges of automated testing inference and workflow orchestration, which have become the primary constraints on delivery speed and quality.
The article explains why fast AI‑generated drafts still demand tedious manual corrections and presents a three‑step automated testing protocol—format locks, logical validation, and redundancy filtering—that shifts quality checks upstream, reducing manual rework by about 80% and error rates below 2%.
During the 2026 Spring Festival promotion, Xiaohongshu replaced manual UI testing with a three‑layer AI‑driven GUI Agent that executed over 43,000 runs across 106 devices and 128 scenarios, achieving 58% automation, 82% AI‑generated case adoption, 68% bug recall, 98% stability and roughly $1 per test case while drastically cutting token costs.
The article shows how AI‑powered "smart mock" decorators let developers describe a scenario in natural language and instantly obtain realistic JSON responses with optional latency, eliminating the need to write hundreds of lines of mock data or maintain separate mock servers.
The article explains how a self‑repairing Agent Harness replaces traditional QA by looping evaluation, triage, automated fixing, verification and AI‑gated canary release, using a three‑judge reviewer, model‑based sampling and six daily engineering tasks to keep AI‑driven products reliable.
The article outlines why rigorous, automated evaluation is essential for AI agents, defines core concepts such as tasks, trials, graders, and frameworks, compares code‑based, model‑based and human graders, and presents an eight‑step roadmap—from early testing to open‑source maintenance—to create reliable, scalable agent assessments.
The article evaluates four popular test data generation tools—Mockaroo, DataFactory, Faker, and DataSynth—across usability, extensibility, data quality, and compliance, providing real‑world case studies and scenario‑based recommendations to help teams choose the right solution.
The article explains how adversarial testing shifts from a user‑centric to an attacker‑centric paradigm, illustrates real‑world cases in finance, autonomous driving and AI, outlines perturbation layers, evaluation metrics, automation pipelines, and three counter‑intuitive principles for effective deployment, highlighting its advantages over conventional testing.
The article outlines a comprehensive, repeatable regression testing framework for TensorRT inference pipelines, covering engine build validation, functional correctness against golden outputs, performance monitoring, common pitfalls, and CI/CD integration to ensure model updates remain both fast and reliable.
The article examines why front‑end testing was once ignored, outlines how AI can automate test creation, proposes a practical three‑layer testing stack (Vitest, Playwright, Chromatic), provides prompt templates and code examples, and offers review guidelines and business arguments for adopting AI‑driven testing.
This article presents seven validated strategies—layered architecture, contract‑driven assertions, configuration‑driven data, smart mocking, impact‑based test selection, OpenAPI‑generated test skeletons, and health‑dashboard monitoring—to dramatically reduce maintenance effort and increase the robustness of automated API tests in fast‑changing projects.
This guide shows how to replace repetitive print or logging calls in automated test suites by creating a reusable @log_test decorator that automatically records start, success, failure, timestamps, and execution time for each test function.
The article argues that in a future where AI generates apps on demand, three core capabilities are essential to manage inherent uncertainty: automated testing from functional specs, runtime safety sandboxes for isolation and permissions, and AI-driven UI generation that replaces traditional UX design, enabling self-compiling, self-verifying, self-sandboxed, self-observing, and self-destructing software.
This article presents a systematic approach to evaluating large language model (LLM) safety by constructing an automated red‑team testing platform that measures prompt jailbreak, privacy leakage, and tool‑execution risks, defines quantitative metrics, compares commercial and open‑source models, and outlines a continuous evolution pipeline for attack samples.
In the era of rapidly advancing large‑model technology, the article outlines the challenges of evaluating data‑centric LLM agents, proposes a three‑layer evaluation framework covering basic capabilities, component‑level checks, and end‑to‑end business impact, and shares practical innovations such as semantic‑equivalence SQL matching, agent‑as‑judge pipelines, and a unified assessment platform.
This technical report details JoyCode Agent’s end‑to‑end pipeline that couples patch generation with fail‑to‑pass and pass‑to‑pass test creation, uses trajectory compression, CSR similarity retrieval, and multi‑agent iterative retries to achieve a 74.6% Pass@1 score on the SWE‑bench Verified benchmark while cutting compute costs by 30‑50%.
This guide explains how to determine whether a web element is visible on the screen during Selenium‑based Java automation, covering the basic isDisplayed() check, an enhanced visibility method that validates size, opacity and CSS visibility, strategies for dynamic, iframe and Shadow DOM elements, and practical best‑practice recommendations.
This article explains how systematic user‑experience testing—covering environment setup, core function benchmarks, and performance monitoring—reveals Distribution’s strengths in multi‑platform compatibility and stability while identifying documentation, configuration, and error‑handling gaps, and recommends tools and continuous improvement practices to enhance the open‑source software distribution platform.
This article explores how AI technologies are reshaping the entire PHP development lifecycle—offering context‑aware code generation, intelligent architecture suggestions, automated test creation, performance optimisation, and end‑to‑end workflow integration—while highlighting real‑world case studies and practical adoption strategies.
This guide compiles the most frequently used kubectl commands for automated testing in Kubernetes, covering context management, service status checks, log retrieval, port forwarding, and practical tips, enabling test engineers to quickly verify deployments, troubleshoot failures, and integrate checks into CI/CD pipelines.
This article details a practical transformation from manual to automated testing using Jest and BDD‑generated HTA, explaining the challenges, technical solutions, mock‑data management, pipeline integration, and measurable results such as higher coverage, lower development cost, and faster release cycles.
The paper presents a systematic testing framework for large‑model interaction capabilities, identifies five unique testing challenges, defines four core abilities (basic dialogue, information extraction, intent recognition, process decision), proposes a semantic‑vector automated verification method, and reports pilot results showing significant efficiency gains and defect detection improvements in a banking context.
Automated testing is essential but challenging for legacy systems; this guide explains testing types, prioritizes unit and integration tests, classifies code for targeted testing, demonstrates refactoring techniques with Java examples, and applies SOLID principles to improve testability and maintainability of aging codebases.
This article details how test development engineers tackled CDN stability challenges by implementing a dual overseas architecture, automated backend interface validation, and a data‑driven front‑end image detection pipeline, achieving an 80% efficiency gain and minute‑level issue detection.
Autogen_demo, built with Streamlit and AI technologies, offers an end‑to‑end solution that parses PDF or Word requirements, stores them in a version‑controlled database, and automatically generates comprehensive functional, boundary, and exception test cases, exportable as Excel, Markdown, or JSON, boosting efficiency for developers, testers, and analysts.
This article presents ten actionable recommendations—including using layered abstractions, prioritizing testability, running tests frequently in CI, adhering to the test pyramid, adopting Page Object and BDD patterns, understanding AI, selecting appropriate tools, and investing in training—to dramatically improve the effectiveness and maintainability of automated testing in software projects.
This article examines the current low self‑testing rates in the frontend team, explains why automated quality‑assurance techniques like code coverage and E2E testing are essential, outlines a comprehensive technical and operational plan—including tooling, rollout, metrics, and future road‑maps—to boost R&D self‑testing ratios and overall delivery quality.
The article introduces an AI‑driven automated testing system that streamlines code self‑testing, improves impact assessment, and generates comprehensive test reports, detailing its modular architecture, workflow, AI test engine, and future enhancements such as UI simulation and AI‑generated unit tests.
The article examines why Claude 4.0 suddenly generates large amounts of code, evaluates the strategic value of training vertical AI models, forecasts visual large‑model adoption in automated testing, and proposes a phased AI‑engineering capability roadmap for teams of different sizes.
This article provides a comprehensive tutorial on Monkey testing for Android applications, covering the complete workflow from environment setup to test execution, monitoring, and analysis.
This article outlines essential automated testing best practices—including clear planning, tool selection, script maintenance, data‑driven approaches, independent test cases, parallel execution, version control, and CI/CD integration—to help teams increase coverage, speed, and software quality.
This article explores the design and implementation of a high‑performance JSON assertion engine for automated testing, detailing its architecture, rule system, core comparison algorithms, visualization features, configuration options, and future enhancements, while providing code examples and performance insights.
The article argues that traditional manual testing is losing relevance as LLM‑powered evolutionary GUI agents—exemplified by AppAgentX—introduce memory chains, action‑evolution mechanisms, multi‑agent collaboration, and RAG‑enhanced knowledge, achieving up to 40% fewer steps, over 50‑point success‑rate gains, and more than 60% faster execution.
The Flow Replay Platform automates traffic‑rule inspection by recording traffic from all environments, letting engineers define jsonPath‑based interface rules that continuously validate pre‑release and production traffic, instantly alerting anomalies, reducing false positives, accelerating release verification, and cutting manual testing effort, as demonstrated by discovered coupon‑related bugs.
This guide demonstrates how to build a Python‑based automated testing framework that reads test cases from a MySQL database, sends HTTP requests, manages case dependencies and parameterization, and records responses, providing a modular structure with database, executor, and main components.
This article outlines a comprehensive automated testing framework for financial systems, covering goals, architecture, tools, processes, measurable outcomes, challenges, and improvement measures to boost test efficiency, coverage, defect detection, and system stability.
This article outlines the key benefits of automated testing frameworks, details a step‑by‑step design process, provides a practical Python‑Selenium example, and showcases real‑world applications in e‑commerce, mobile, and financial systems, illustrating how such frameworks accelerate development and improve software quality.
The article describes a reflection‑based workflow that lets an AI iteratively generate, validate, and refine Go unit tests until 70 %+ code coverage is reached, using coverage‑out parsing, role‑based prompts, and automatic error correction, enabling fully autonomous test creation that can be integrated into CI pipelines to dramatically improve developer productivity.
This article guides functional testers on transitioning to automated testing with Python, covering core concepts, a step‑by‑step learning plan, and ten practical code snippets ranging from unittest and pytest to Selenium web and API automation.
The article describes Vivo's Engineering Efficiency team's VCR platform, built on GitLab and Gerrit, detailing the design and implementation of code‑review plugins, challenges faced, architectural integration, customization techniques, and future directions toward automated and AI‑enhanced code review.
YouZan implemented a BackstopJS‑based visual regression testing platform that automatically compares pre‑release and production pages, managing cases, scenarios, tags and selectors, which now runs over 1,000 tests, saves roughly 300 person‑days per year, has uncovered more than 100 e‑commerce issues, and is being extended to mini‑programs, React/Vue component checks, and traffic‑playback UI monitoring.
This article examines the open‑source AutoDev AI‑driven programming tool, outlining its three maturity stages—from chat‑based assistance to fully automated generation, verification, and execution—and detailing new features such as automatic refactoring, API test execution, bulk test creation, and batch prompting to improve developer productivity.
Unmanned MOCK automatically generates intelligent, context‑aware mock responses for downstream services in end‑to‑end tests by collecting sub‑call data, extracting knowledge, and applying dynamic rules, so failures in downstream systems are isolated, raising test success rates toward near‑100 % without manual mock configuration.
The article introduces AREX, Ctrip’s open‑source traffic recording and replay platform, detailing its background, design decisions, technical challenges such as cross‑thread collection and non‑idempotent interfaces, deployment strategies, best‑practice integration with CI/CD pipelines, and the measurable benefits achieved across thousands of applications.
Facing rapid business expansion, Huolala’s quality assurance team tackled redundant code, high regression costs, and lack of coverage metrics by designing an intelligent automated testing framework that analyzes effective code, provides smart test case recommendations, visualizes progress, and integrates monitoring, resulting in significant coverage improvements and efficiency gains across services.
This article explains why recording and replaying HTTP interactions with Betamax is essential for automated testing, outlines its benefits such as consistency, efficiency, error reduction, regression support, and provides multiple advanced code examples for custom recording modes, request filtering, cassette control, context‑manager usage, and handling dynamic content.
The article explains how to implement online automated inspection in production environments, detailing its definition, step‑by‑step rollout, key scenario selection, task creation, rule validation, execution controls, fault response, and post‑inspection review to maintain system quality and reduce risk.
This article explains how to implement online traffic replay for Python automated testing by parsing logs, storing request data in a local MySQL database, analyzing user information with data‑analysis libraries, and locally persisting URL, method, headers, and body details using JSON files, complete with sample code.
This article presents a self‑developed automated testing platform for backend services that achieves full test‑case coverage with low maintenance cost by unifying HTTP and RPC access, introducing a parameter‑pool, integrating JSON Schema and JSONPath validation, and leveraging traffic‑driven test‑case generation and scheduling.
This article outlines key principles and best practices for designing a scalable Python automated testing architecture, covering test strategy definition, framework selection, modular and keyword‑driven design, test data management, error handling, reporting, CI integration, and provides a complete example implementation.
This article introduces five common Python automated testing scenarios—unit testing, API testing, web UI testing, performance testing, and database testing—providing clear explanations and complete code examples to help developers improve software quality and stability.
This article explains how to calculate and improve the ROI of automated testing, introduces the 3KU rule for prioritizing test layers, discusses tool selection, script reuse, left‑ and right‑shift testing, and provides metrics and pipeline designs to achieve high‑impact test automation.
This article demonstrates how to use Playwright with Pytest to run automated browser tests, generate detailed HTML reports, and provides best‑practice guidelines for building a comprehensive, maintainable test suite for web applications, covering setup and teardown procedures.
This article presents a traffic‑driven quality assurance framework for an Elasticsearch‑based search service, detailing active and inactive code protection strategies, automated scenario generation from Dubbo logs, template fingerprinting, de‑duplication, expected result pools, and validation rules to ensure comprehensive test coverage and reliable regression.
This article describes the design, implementation, and operational management of a full‑process automated testing platform for airline ticket systems, covering background analysis, goals, measurement metrics, platform architecture, case handling, result verification, quality gates, and future improvement plans to enhance system stability and quality.
This article details how a rapidly expanding audio service built a lightweight, modular testing framework called FAST, defined core scenario‑based testing concepts, and implemented an automated pipeline that reduced manual effort, cut costs, and improved release quality across multiple teams.
This article evaluates the new Copilot Chat feature in VS Code 1.80, showing how to install it, use its slash commands for code explanation, bug fixing, test generation, and project scaffolding, while highlighting its strengths, current limitations, and practical impact on developer productivity.
This article presents the design, implementation, and operational experience of AREX, a Java‑agent based record‑and‑replay testing platform that captures real production traffic and data to enable fast, cost‑effective automated regression testing for complex backend services.
JSONCompareUtils, a full‑JSON comparison component added to the Artemis framework, lets automated tests recursively compare large JSON responses via a Maven‑provided library with Java and Python wrappers, streamlining service‑splitting, BC‑traffic, and cross‑environment validation while dramatically cutting manual verification effort and speeding regression testing.
This article presents a comprehensive backend automated testing framework that unifies HTTP and RPC access, introduces a parameter‑pool concept, leverages JSON Schema and JSONPath for validation, and outlines coverage metrics, test case generation, discovery, and continuous improvement to achieve near‑100% API test coverage.
This article details the design and implementation of a self‑built automated testing platform for backend services that achieves near‑full test‑case coverage by unifying HTTP/RPC access, introducing a parameter‑pool, leveraging JSON Schema and JSONPath, and automating case generation and promotion.
The article presents a language‑agnostic, low‑maintenance automated backend interface testing system that unifies HTTP and RPC calls, uses a parameter pool, JSON Schema and JSONPath for assertions, generates test cases from live traffic, measures coverage, and continuously updates suites to achieve near‑full coverage.
This article describes Qunar's use of traffic recording and replay technology for fault isolation, performance optimization, and upgrade migration, detailing the evolution of their Java‑based agents, the end‑to‑end testing workflow, full‑link stress‑test integration, and the measurable business and performance outcomes.
This article explains how to use Python's unittest and pytest frameworks to create automated test cases, build test suites, and explicitly control the execution order of tests, providing code examples and a summary of best practices for effective test management.
This article introduces RunnerGo's automated testing features, detailing plan management, report management, scene import, task configuration, and how to view multi‑dimensional test results, while providing links to the project's GitHub and Gitee repositories for further exploration.
This article analyzes how 33% of data‑center defects stem from big‑data scheduling errors and proposes an automated testing system that monitors data metric integrity, accuracy, and timeliness through interface automation, visual dashboards, alert bots, and a custom ORM framework to dramatically improve ROI.
This article explains what unit testing is, provides C# code examples for a simple calculator, and discusses four reasons why automated unit tests should run in CI/CD pipelines to improve development speed, prevent regressions, and protect code written by other developers.
This article analyzes the challenges of automating test coverage for a large‑scale MMORPG by classifying representative functional modules, outlining targeted testing strategies for different module characteristics, and proposing framework and script‑library optimizations to efficiently increase coverage and maintainability.
Test Case Designautomated testingframework optimization
Moonbox is an open‑source, JVM‑Sandbox‑based platform that records and replays traffic to generate automated regression tests, offering visual management, rich plugins, multiple deployment options, high performance and safety features, and aims to simplify testing for developers while fostering community contributions.
This article discusses common challenges in automated testing and provides strategies to address them, including gaining management approval, selecting appropriate tools, determining a starting strategy, and setting realistic expectations for automation.
This article explains what microservices are, defines microservice testing, outlines automated testing levels such as unit, contract, integration, and UI testing, and presents strategies, best practices, and tool recommendations for implementing effective automated testing in microservice architectures.
Learn how to install and configure the Apifox Helper plugin in IntelliJ IDEA to automatically generate, sync, and debug Java Spring API documentation, offering a zero‑intrusion alternative to Swagger with built‑in mock and testing features.
The article explains why automated testing is essential for continuous testing, describes the rise of layered automation approaches, discusses quality‑risk management concepts such as Heinrich's law, and outlines the evolution from the classic testing pyramid to emerging models driven by AI and equal investment across test levels.
The article describes JD's AI‑driven UI traversal framework that combines traditional view‑tree dumping, image‑based segmentation, OCR and YOLO detection to automatically explore, assert and handle dynamic elements across Android, iOS, H5 and mini‑programs, improving testing efficiency and reducing production incidents.
The article explains why shifting from manual to automated testing is essential in fast‑changing agile environments, outlines common migration challenges, and details the advantages such as faster testing, cost reduction, improved reliability, and stronger CI/CD integration.
To cope with Meituan Waimai’s exploding test case volume and diverse technology stacks, the AlphaTest platform was built to provide zero‑learning‑cost, low‑maintenance automated testing that records, simulates environments, uses multi‑modal element locating, cross‑App playback, and AI‑driven image assertions, now covering 70 % of regression cases across native, Mach, React Native, mini‑program and H5.
Exploring Go 1.18, the article walks readers through its three flagship additions—generics, fuzzing, and workspaces—detailing installation, underlying implementation concepts, practical code examples, and how each feature enhances type safety, automated testing, and multi‑module dependency management for modern backend development.