Comprehensive Testing Practices for Large‑Scale Platform Migration

The trading platform completed several large‑scale migrations this year, moving from the product domain to the order domain and finally the order return (RMA) domain. Although the systems are now stable in production, the testing phase revealed challenges that are summarized here for future reference.

The testing scope covered data consistency, interface logic, functional, data synchronization, and performance testing, with pre‑deployment rehearsals and rollback drills involving all business stakeholders.

Data Consistency Testing

Ensuring data consistency is critical because the migration must be transparent to callers. The team compared .NET and Java API request parameters, response values, and database records using an automated Python script that leverages decorators to switch environments and reuse test cases.

def deco_compare(env=1, db_compare=False, ex='', db_ex=''):
    # decorator implementation ...
    def deco_resp(func):
        @wraps(func)
        def wrapper(*args, **kw):
            print('执行用例 %s():' % func.__name__)
            # ... logic based on env and db_compare ...
            return func(*args, **kw)
        return wrapper
    return deco_resp

During testing, environments 3 and 4 were used, and the json_tools library compared two JSON payloads, reporting differences in values, types, and field names.

Interface Logic Testing

Most functionalities of the product, order, and RMA systems are exposed via APIs, making interface logic testing a core focus. For the order system, tests covered order creation, modification, delivery, revenue confirmation, and queries, encompassing 14 product types, 24 order types, and 8 promotion schemes.

To manage this complexity, the team parameterized scripts to pass different data sets, extracted common business functions (e.g., basic order creation, revenue verification), and reused them across test flows.

Functional Testing

Although the platform has few visible functional requirements, functional testing remains vital for end‑users. Test points were defined in collaboration with product owners, and no major issues were observed after launch.

Data Synchronization Testing

The migration also involved moving code from .NET to Java and databases from SQLServer to MySQL. Full‑volume data sync (DataX) and incremental reverse sync (Yugong) ensured no data loss and supported rollback scenarios. Log‑analysis scripts parsed synchronization logs to surface any anomalies.

Performance Testing

Performance testing was essential ahead of high‑traffic events like Double‑Eleven. The team used a ThreadPool library to generate concurrent requests, as shown below.

import threadpool

def run_thread_pool(self, threads_num, request_num, target_request):
    """Create a thread pool.
    :param threads_num: number of threads
    :param request_num: total requests
    :param target_request: function to execute
    """
    pool = threadpool.ThreadPool(threads_num)
    requests = threadpool.makeRequests(target_request, request_num)
    [pool.putRequest(req) for req in requests]
    pool.wait()
    return wrapper

By configuring thread count and request volume, the team achieved realistic concurrency testing.

Business Acceptance and Rollback Rehearsals

Before go‑live, the team conducted acceptance rehearsals with major business units (e.g., 大网校, 享互), simulating end‑to‑end scenarios and involving finance, customer service, and BI teams. Issues uncovered during rehearsals were fixed, boosting confidence in the production rollout. A rollback rehearsal was also performed, ensuring preparedness despite the smooth launch.

Challenges and Solutions

Limited business knowledge: testers relied on product and development support, using flowcharts, sequence diagrams, and unit tests to quickly grasp domain logic.

Resource constraints: with only four testers for the order system, the team prioritized early automation, synchronized test preparation with development, and sought external help when needed.

Technical decisions: after evaluating several approaches, the team settled on decorator‑based data comparison, which, although time‑consuming, provided a solid verification foundation.

Outcomes

The migration was completed ahead of schedule (order system downtime 38% of the planned window, RMA system one day early). Automated scripts dramatically reduced manual effort, improved accuracy, and accelerated familiarity with business rules such as promotion calculations and order revenue distribution.

Overall, the testing framework and new techniques established during this migration will guide future large‑scale projects on the platform.