Scaling a Consumer‑Goods Business Platform to 10M Daily Calls with SpringBoot, Kong, and K8s

Business Architecture

The platform serves as a middle‑layer for managing customer assets such as cards, coupons, and other virtual items. It exposes standard RESTful APIs to internal apps, mini‑programs, and partner channels (e.g., banks, Alibaba). Daily request volume is about 7 million calls, peaking over 10 million during major events.

Technical Architecture

The solution uses a SpringBoot‑based microservice architecture, containerized with Kubernetes (K8s) and exposed through Kong API Gateway.

Kong API Gateway

Kong provides an enterprise‑grade gateway with sub‑millisecond latency (Nginx/OpenResty), single‑node throughput of ~25 K TPS, and built‑in authentication, rate‑limiting, request transformation, logging, and analytics. APIs are registered in Kong, certificates are added, and Auth Keys are generated for partners.

Application Architecture

Four containerized services compose the system:

Asset Service – asset‑related APIs.

Asset Consumer Service – RabbitMQ listener for asynchronous asset processing.

Console Service – management and operational APIs for the admin console.

Console Frontend – Vue.js UI.

Custom MyBatis Configuration

MyBatis is used for data access. A custom SqlSessionFactory bean supports multiple databases and pagination via PageHelper.

@Bean
public SqlSessionFactory sqlSessionFactory(DataSource dataSource) throws Exception {
    SqlSessionFactoryBean sfb = new SqlSessionFactoryBean();
    sfb.setDataSource(dataSource);
    sfb.setVfs(SpringBootVFS.class);
    Properties props = new Properties();
    props.setProperty("dialect", dataConfiguration.getDialect());
    props.setProperty("reasonable", String.valueOf(dataConfiguration.isPageReasonable()));
    PageHelper pagePlugin = new PageHelper();
    pagePlugin.setProperties(props);
    sfb.setPlugins(new Interceptor[]{pagePlugin});
    ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
    sfb.setMapperLocations(resolver.getResources("classpath*:mappers/" + dataConfiguration.getDialect() + "/*.xml"));
    sfb.setTypeAliasesPackage("com.xxx.bl.core.data.model");
    SqlSessionFactory factory = sfb.getObject();
    factory.getConfiguration().setMapUnderscoreToCamelCase(true);
    factory.getConfiguration().setCallSettersOnNulls(dataConfiguration.isCallSettersOnNulls());
    return factory;
}

Logback Logging

Logback is configured per Spring profile to set different log levels and appenders. Spring Cloud Sleuth adds a traceId to all logs for end‑to‑end tracing. Asynchronous logging reduces I/O blocking.

<springProfile name="stg">
    <root level="error">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="SAVE-ERROR-TO-FILE-STG"/>
    </root>
    <logger name="org.xxx" level="error" additivity="false">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="ASYNC-SAVE-TO-FILE-STG"/>
    </logger>
</springProfile>
<springProfile name="prod">
    <root level="error">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="SAVE-ERROR-TO-FILE-PROD"/>
    </root>
    <logger name="org.xxx" level="error" additivity="false">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="ASYNC-SAVE-TO-FILE-PROD"/>
    </logger>
</springProfile>

SSL Encryption

SSL is enabled by adding a JKS certificate to the classpath and configuring SpringBoot properties. Passwords can be encrypted with Jasypt or managed via Azure Key Vault / CyberArk Conjur.

ssl:
  enabled: true
  key-store: classpath:xxx.net.jks
  key-store-type: JKS
  key-store-password: RUIEIoUD
  key-password: RUIEIoUD
  require-ssl: true

Synchronous & Asynchronous Service Switching

A three‑node RabbitMQ cluster handles asynchronous processing. Requests are classified by an API code; synchronous requests are processed directly, while asynchronous ones are queued to RabbitMQ and consumed by a dedicated component, enabling traffic‑spike mitigation.

Dockerfiles

SpringBoot services use a simple Dockerfile based on OpenJDK 11; the Vue frontend uses an Nginx‑based image with SSL certificates.

FROM openjdk:11-jre
ARG JAR_FILE=console-service/build/libs/*.jar
COPY ${JAR_FILE} app.jar
EXPOSE 9002 9003
ENTRYPOINT ["java", "-jar", "/app.jar"]

FROM nginx:stable-alpine
COPY /dist /usr/share/nginx/html/console
COPY nginx.conf /etc/nginx/nginx.conf
ARG KEY_FILE=stg.xxx.net.key
ARG PEM_FILE=stg.xxx.net.pem
COPY ${KEY_FILE} /etc/ssl/certs/cert.key
COPY ${PEM_FILE} /etc/ssl/certs/cert.pem
EXPOSE 80
CMD ["nginx", "-c", "/etc/nginx/nginx.conf", "-g", "daemon off;"]

Database Architecture

To handle billions of account and transaction records, the system uses MyCat as a proxy for MySQL, implementing master‑slave replication with read‑write separation. A 1‑master‑2‑slave‑1‑backup topology provides high availability.

Half‑sync replication ensures data consistency between master and slaves.

Automatic failover switches to a slave when the master fails.

After master recovery, the original master becomes a slave before being promoted again.

Sharding Strategy

Vertical sharding separates data by domain (account, asset, transaction) into different databases. Horizontal sharding splits large tables (e.g., transaction logs) by date, allowing independent scaling and easier archival of cold data.

Cold & Hot Data Management

Hot data (frequently accessed) uses query cache and Redis for low‑latency reads.

Cold data (historical transactions) is archived to separate storage or served from a standby replica.

DevOps & K8s Deployment

A Jenkins‑based CI/CD pipeline pulls code from GitHub, builds the backend with Gradle and the frontend with npm, creates Docker images, and deploys them to a K8s cluster. Resource requests (e.g., 2 CPU / 4 Gi) and replica counts are defined per service.

Rolling updates and horizontal pod autoscaling based on CPU metrics ensure zero‑downtime scaling during peak hours.

Configuration Management

SpringBoot configuration is externalized via a K8s ConfigMap. The application JAR excludes the default application.yml, and the ConfigMap is mounted at runtime.

-spring.profiles.active=prod
-spring.config.location=/config/application.yml

Monitoring

The ELK stack (Elasticsearch, Logstash, Kibana) collects, processes, and visualizes logs. Dynatrace provides APM, auto‑detects services, and offers detailed performance dashboards.

Reflections & Lessons Learned

MyCat sharding introduces code intrusiveness; future projects may consider NewSQL solutions like TiDB.

JVM tuning (e.g., moving to JDK 13 with ZGC) mitigated occasional long‑GC pauses.

Introducing a local Guava cache reduced Redis pressure.

Investing in a shared application framework enabled low‑code development and higher developer productivity.

Standardizing patterns across teams improved code quality and reduced bugs.

The case study demonstrates how a combination of SpringBoot microservices, Kong gateway, MyCat sharding, Docker, Kubernetes, and robust DevOps practices can support a high‑throughput, enterprise‑grade business platform.