How JD Daojia Built a Robust Continuous Integration Pipeline

Continuous Integration Concept

Continuous integration (CI) is a software development practice where team members frequently merge their work, typically at least once a day, triggering automated builds, tests, and deployments to catch integration errors early and accelerate delivery of cohesive software.

Value of Continuous Integration

1. Risk Reduction

Early detection of defects lowers fixing costs and reduces release‑time risk by continuously monitoring software health.

2. Eliminate Repetitive Work

CI automates compilation, deployment, and testing, minimizing manual intervention.

3. Deploy Anywhere, Anytime

Small code changes can be integrated continuously; any failure instantly notifies the team for rapid remediation.

Why CI Matters for JD Daojia

Rapid iteration cycles and fast service deployment posed challenges for testing and development. JD Daojia introduced CI to address three main problems:

Poor test quality due to compressed development cycles.

Long testing cycles that increase cost when code quality degrades.

High amount of repetitive manual work.

JD Daojia’s CI Implementation

Code is committed to Git. A static analysis job runs first, followed by unit‑test and coverage jobs, then compilation, packaging, and deployment to target servers. UI, JSF, and mobile (APP) automated tests run subsequently; failures are analyzed and fixed, with optional functional testing and coverage collection.

Static code scanning enforces fixing of blocking and high‑severity bugs. All systems achieve fully automated compile, package, and deployment, with continuous addition of test cases and coverage monitoring.

CI Architecture Overview

CI Server Architecture

Evolution of Test Environments

Stage 1: Manual compile, package, and deploy – resolved by introducing Jenkins for automated builds.

Stage 2: Disk space exhaustion and service crashes – addressed with scheduled cleanup, service monitoring, and auto‑restart.

Stage 3: Distributed services failing to communicate due to inconsistent local configuration – solved by maintaining separate test environments, syncing configurations, and using automated health checks.

Static Code Scanning

Steps to set up SonarQube scanning in Jenkins:

Configure SonarQube environment.

Install SonarQube plugin in Jenkins.

Run Sonar analysis after the static‑scan job.

Code Coverage Collection

Unit Test Coverage Setup

Configure pom.xml with Sonar Maven plugin, Surefire plugin, and JaCoCo plugin. Example snippet:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>sonar-maven-plugin</artifactId>
  <version>1.0</version>
</plugin>

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.9</version>
  <configuration>
    <skip>false</skip>
  </configuration>
</plugin>

<plugin>
  <groupId>org.jacoco</groupId>
  <artifactId>jacoco-maven-plugin</artifactId>
  <version>0.7.4.201502262128</version>
  <executions>
    <execution>
      <id>default-prepare-agent</id>
      <goals><goal>prepare-agent</goal></goals>
    </execution>
    <execution>
      <id>default-report</id>
      <phase>prepare-package</phase>
      <goals><goal>report</goal></goals>
    </execution>
    <execution>
      <id>default-check</id>
      <goals><goal>check</goal></goals>
      <configuration>
        <rules>
          <rule implementation="org.jacoco.maven.RuleConfiguration">
            <element>BUNDLE</element>
            <limits>
              <limit implementation="org.jacoco.report.check.Limit">
                <counter>COMPLEXITY</counter>
                <value>COVEREDRATIO</value>
                <minimum>0.00</minimum>
              </limit>
            </limits>
          </rule>
        </rules>
      </configuration>
    </execution>
  </executions>
</plugin>

Run mvn clean install to generate coverage reports.

Functional Test Coverage Setup

Insert listener parameters into startup scripts, dump listener files, and use ANT scripts to generate final HTML reports. Configuration steps are illustrated with screenshots.

Automated Testing

UI Automation – integrated into Jenkins using Selenium, TestNG, and Maven.

HTTP API Automation – built with JMeter, Ant, and Jenkins. Sample Ant build file:

<?xml version="1.0" encoding="UTF-8"?>
<project name="ant-jmeter-test" default="all" basedir=".">
  <tstamp/>
  <property name="jmeter.home" value="D:\program\apache-jmeter-2.13"/>
  <property name="jmeter.result.jtl.dir" value="results"/>
  <property name="jmeter.result.html.dir" value="results"/>
  <property name="ReportName" value="maincase"/>
  <property name="jmeter.result.jtlName" value="${jmeter.result.jtl.dir}/${time}-${ReportName}.jtl"/>
  <property name="jmeter.result.htmlName" value="${jmeter.result.html.dir}/${time}-${ReportName}.html"/>
  <property name="jmeter.result.index" value="${jmeter.result.html.dir}/index.html"/>
  <target name="all">
    <antcall target="clean"/>
    <antcall target="test"/>
    <antcall target="report"/>
  </target>
  <target name="clean">
    <delete>
      <fileset dir="${jmeter.result.jtl.dir}" includes="*.jtl"/>
    </delete>
  </target>
  <target name="test">
    <taskdef name="jmeter" classname="org.programmerplanet.ant.taskdefs.jmeter.JMeterTask"/>
    <jmeter jmeterhome="${jmeter.home}" resultlog="${jmeter.result.jtlName}">
      <testplans dir="jmeter" includes="*.jmx"/>
    </jmeter>
  </target>
  <target name="report">
    <xslt in="${jmeter.result.jtlName}" out="${jmeter.result.htmlName}" style="${jmeter.home}/extras/jmeter-results-detail-report_21.xsl"/>
    <delete file="${jmeter.result.index}"/>
    <copy tofile="${jmeter.result.index}" file="${jmeter.result.htmlName}"/>
    <copy todir="${jmeter.result.html.dir}">
      <fileset dir="${jmeter.home}/extras">
        <include name="collapse.png"/>
        <include name="expand.png"/>
      </fileset>
    </copy>
  </target>
</project>

Mobile App Automation – uses Appium for Android and iOS because it works without source code dependency.

Challenges and solutions:

Unstable networks cause high failure rates – a retry listener attempts three times before marking a case as failed.

Difficult to pinpoint UI failures – added screenshot and extensive logging to aid debugging.

Slow execution – Android tests run in parallel across multiple services.

Future Plans

Continuously increase automation coverage to reduce repetitive work.

Integrate stress regression testing to ensure performance before release.

Provide additional testing tools to improve efficiency.

For more technical articles, follow the New Dada Technology public account.