Big Data Performance Testing: Objectives, Timing, Steps, Tools, and Optimization

Purpose of Big Data Performance Testing

1. Conduct performance regression of big data components by comparing new and old versions during upgrades.

2. Establish performance baselines after releasing new versions or production environments to provide measurable reference standards for other test scenarios and tuning processes.

3. Compare multiple release versions to supply reference data for PoC testing.

4. Support PoC testing to draw conclusions and select appropriate solutions based on business models, requirements, or customer needs.

5. Perform performance testing on the client side to meet required performance standards and satisfy user demands.

Timing of Performance Testing

1. When a new version is launched.

2. When a new environment or host is introduced.

3. When a new region is opened.

4. During PoC testing.

5. For dedicated performance testing projects.

Steps of Performance Testing

1. Clarify testing objectives, including test scenarios, cluster size and specifications, data volume, data format, compression algorithms, etc. For version iteration tests, align cluster specifications with historical versions; for PoC tests, define customer scenarios; for vendor tests, match vendor cluster scale.

2. Apply for host resources and define the testing schedule.

3. Set up the runtime environment and monitoring tools.

4. Collect performance metrics such as bandwidth, disk I/O, CPU, memory, and other indicators.

5. Execute tests using tools like nmon or other system monitoring utilities to record metric changes and identify bottlenecks for subsequent tuning.

6. Adjust and optimize based on test results, iterating performance tests as needed.

7. Produce a performance testing report.

Big Data Component Testing Tools and Methods

In addition to mainstream big data engines, technologies such as HBase are also relevant.

Beyond mainstream testing tools like HiBench, Yahoo's big data testing suite is also available.

Useful repositories: https://github.com/Intel-bigdata/HiBench , https://github.com/elastic/rally , https://github.com/yahoo/streaming-benchmarks , https://github.com/brianfrankcooper/YCSB

Big Data Performance Tuning

1. Data skew is a common issue in big data; refer to the official documentation of the respective component.

2. Consult industry case studies for best‑practice guidance.

Related Big Data Tests

Benchmark testing – single‑user, single‑transaction tests to gauge system handling of isolated requests.

Load testing – gradually increase system load to observe performance changes.

Stability testing – apply continuous business pressure for 24/7 operation to verify system stability.

Functional testing – especially when selecting OLAP engines, verify support for standard SQL features such as UPDATE, DELETE, WITH, EXCEPT, INTERSECT, etc.

Performance requirements – CPU, memory, disk I/O, network utilization should stay below 80%; response times for 90% of reads/writes/exports/imports should be under 3 seconds, with less than 10% exceeding 5 seconds.

Test cases – e.g., benchmark Hadoop and Spark with varying data sizes (100 GB, 500 GB, 1 TB) for read, write, export, and import operations.

Additional scenarios include parallel read/write mixed tests at different data volumes and 7 × 24 hour stability tests.

Observed metrics:

1. CPU usage
2. Memory usage
3. I/O
4. Network
5. Response time
6. Other indicators