Data Testing | BestHub

Beijing SF i-TECH City Technology Team

Jan 16, 2024 · Big Data

Big Data Testing: Methods, Tool Selection, and Practical Implementation with Datacompy

This article introduces big data testing concepts, outlines common testing methods, evaluates the Python library Datacompy against alternatives, and details a practical implementation for large-scale data migration and validation, including configuration, volume comparison, content verification, and performance optimizations such as sorting and multithreading.

Big DataData TestingPython

0 likes · 10 min read

Big Data Testing: Methods, Tool Selection, and Practical Implementation with Datacompy

DevOps

Oct 25, 2023 · Big Data

An Introduction to Big Data: Origins, Definitions, 5V Characteristics, Applications, Hadoop Architecture, and Testing Strategies

This article provides a comprehensive overview of big data, covering its origins, definitions, 5V characteristics, data formats, real‑world applications, Hadoop architecture, testing challenges, functional and performance testing strategies, and the skills required for effective big data testing.

5V CharacteristicsBig DataData Formats

0 likes · 35 min read

An Introduction to Big Data: Origins, Definitions, 5V Characteristics, Applications, Hadoop Architecture, and Testing Strategies

360 Quality & Efficiency

Aug 12, 2022 · Big Data

Testing Plan and Efficiency Strategies for a Data Download Refactoring Project

This article outlines the testing plan for a data download refactoring project involving over 400 metrics, describes automated CSV comparison scripts, evaluates single‑process, multithreaded, and multiprocess approaches with shared memory, and provides practical recommendations for improving verification efficiency and performance.

CSV comparisonData TestingPython multiprocessing

0 likes · 8 min read

Testing Plan and Efficiency Strategies for a Data Download Refactoring Project

Fulu Network R&D Team

Sep 21, 2020 · Big Data

Data Development and Testing: Process, Key Concerns, and Quality Monitoring

This article outlines the data development lifecycle, distinguishes it from application development, details the responsibilities and focus areas for data testers, and presents a comprehensive end‑to‑end quality monitoring and alert system for big‑data pipelines.

Big DataData TestingETL

0 likes · 14 min read

Data Development and Testing: Process, Key Concerns, and Quality Monitoring

Liulishuo Tech Team

Oct 22, 2017 · Big Data

Data-CI: A SQL-Based Data Unit Testing Framework for ETL

The article introduces data-ci, a SQL‑driven unit testing framework that lets engineers write, organize, and automate data validation tests for ETL pipelines, providing assertions, failure callbacks, coverage reporting, and CI integration to improve data quality and reliability.

Big DataData TestingETL

0 likes · 9 min read

Data-CI: A SQL-Based Data Unit Testing Framework for ETL