The article calculates the time and cost of transferring 1 PB of data between Nanjing and Beijing via various network links and shows that shipping hard‑drive‑filled trains can deliver the data in about seven hours at a fraction of the bandwidth cost.
Serialization in C++ bridges in‑memory objects and persistent or network formats, addressing challenges like byte order, data type compatibility, and complex structures, while the article reviews core principles, binary and text formats, major libraries such as Protobuf, Cereal, Cista++, ThorsSerializer, and offers guidance on selecting and using them effectively.
This article breaks down zero‑copy data transfer techniques—write+read, mmap+write, sendfile, sendfile + SG‑DMA, and splice—showing how they reduce CPU copies and context switches to boost I/O performance in modern operating systems.
This guide explains two primary techniques—using the Navigation component and the router object—to pass parameters between pages in HarmonyOS NEXT, providing step‑by‑step code examples, practical scenarios, and tips for choosing the appropriate method.
The article examines how long it would take to transfer a 1,000 TB dataset from Nanjing to Beijing using various network options—home broadband, enterprise data lines, and internet lines—calculates the required time and cost, and ultimately proposes shipping hard drives by high‑speed rail as the fastest, cheapest solution.
An interview question explores the fastest way to transfer a 1,000 TB dataset from Nanjing to Beijing, comparing high‑speed network links, their costs and transfer times, with a practical, low‑cost solution of physically shipping hard‑drive containers by high‑speed rail.
This article explains how zero‑copy technology eliminates redundant memory copies and context switches during data transmission, compares traditional read‑send workflows with zero‑copy approaches such as Linux sendfile/splice and Java's FileChannel.transferTo, and discusses performance benefits and practical considerations.
This article explains how Go's native structs and interfaces can replace traditional Data Transfer Objects, offering a more concise, performant, and maintainable way to move data across application layers while highlighting best‑practice patterns and potential pitfalls.
The DTLE 4.23.04.0 release introduces numerous MySQL‑focused enhancements and bug fixes—including timestamp handling, UUID processing, transaction retry logic, UTF‑8 support, and API improvements—while providing download links, repository information, and recommended reading for users of this open‑source data transfer component.
This article provides an in‑depth examination of InfiniBand, covering its rapid development as a high‑bandwidth, low‑latency interconnect technology, the InfiniBand Trade Association, detailed packet structures, layered architecture, switching mechanisms, and a comparative analysis with Ethernet, highlighting its advantages for high‑performance computing.
DTLE 4.22.11.0 introduces major job configuration changes, new log settings, a TwoWaySync parameter for bidirectional replication, beta Kafka credentials, and fixes for Decimal column defaults, while providing updated documentation links and download resources for the open‑source MySQL data transfer component.
The DTLE 4.22.07.2 release introduces performance‑boosting pipeline enhancements for large transactions, continuous insert optimizations, data chunking for massive loads, a new DumpEntryLimit setting, and bug fixes, while providing repository links, documentation, and recommended articles for further exploration.
DTLE 4.22.01.0, an open‑source MySQL‑focused data transfer component, has been released, introducing Oracle‑MySQL incremental DDL/DML sync, enhanced WHERE clause support, initial Kafka schema‑change handling, and numerous bug fixes including binlog relay issues, character‑set handling, and CREATE TABLE … LIKE renaming problems.
The DTLE 3.21.11.0 release introduces MySQL‑specific enhancements such as partitioned table creation, geometry type support, multi‑threaded replay with unique columns, and large‑transaction handling, while also fixing an alter‑table rename issue, and provides download links and related reading resources.
The DTLE 3.21.10.0 release introduces foreign‑key support, a new job parameter for foreign key checks, cascade operations, handling of large‑memory transactions, and various bug fixes and refactorings, providing a more robust MySQL data‑transfer solution.
The article provides a comprehensive technical overview of InfiniBand, covering its history, standards, architecture layers, packet format, performance advantages, and a detailed comparison with Ethernet, highlighting why it has become the preferred high‑speed interconnect for HPC workloads.
This article explains the purpose of I/O buses, describes common external interfaces like USB, outlines internal bus types such as chip, system, data, address, and control buses, and details various bus arbitration methods with their advantages and drawbacks.
The DTLE 3.21.01.0 release introduces Kafka batch‑send parameters, support for rename‑table DDL, empty transactions for skipped DDL/DCL, and several bug fixes, while providing repository links, documentation, and installation tips for this MySQL‑focused data‑transfer component.
The DTLE 3.20.10.0 release introduces full and incremental job status output, a reschedule option to prevent unlimited retries, code refactoring, and provides repository, documentation, and download links for the open‑source MySQL data‑transfer component, followed by a light‑hearted personal note.
This article explains the zero‑copy data transfer technique, compares it with traditional read/write approaches, shows Java NIO code examples, and discusses its use in high‑performance systems such as Kafka and Spark, highlighting the reductions in context switches and memory copies.
Version 3.20.09.0 of the open‑source DTLE data‑transfer component for MySQL has been released, introducing replication‑delay and memory‑usage monitoring with Prometheus, providing configuration examples and Docker commands, and fixing incremental serialization, CPU usage, and uppercase‑where clause handling.
This article walks through integrating Spring Boot with Netty and using Google Protobuf for binary data exchange, covering Protobuf basics, Maven setup, code generation, server and client implementations, heartbeat handling, reconnection logic, and testing results, all with complete Java examples.
The article explains the rapid evolution of the PCIe bus standard—from the widely used PCIe 3.0 through PCIe 4.0 and 5.0 up to the upcoming PCIe 6.0—detailing version milestones, bandwidth improvements, signaling changes, backward compatibility, and its impact on hardware and AI workloads.
This article thoroughly explains the NVMe over Fabrics architecture, covering its encapsulation format, IO transmission steps, connection management, command extensions, discovery services, and the Linux reference implementation for RDMA and Fibre Channel transports.
The DTLE 2.19.11.0 release notes detail the project's purpose as a MySQL‑focused data transfer component, outline new features such as binlog relay and MySQL 8.0 support, list numerous bug fixes, and provide links to documentation and download resources for developers.
The DTLE 2.19.07.0 release introduces cross‑data‑center transfer, cloud synchronization, data aggregation, subscription, bidirectional replication, performance improvements such as reduced CPU usage and tracing, and a series of bug fixes including connection leaks, NATS address handling, large‑transaction handling, and MySQL 8.0 compatibility warnings.
The DTLE 2.19.05.0 release notes introduce the enterprise‑grade open‑source distributed data‑transfer component, detail a new job‑failure hint feature, and enumerate several bug fixes including table‑name case handling, mapping job failures, Docker startup issues, and Kafka unknown table structures.
This article explains the traditional copy‑based data transmission process, introduces the zero‑copy technique—including basic sendfile(), scatter/gather DMA and mmap support—shows how it reduces context switches and copies, and demonstrates its practical use in Kafka and Spark for high‑throughput workloads.
The DTLE 2.19.03.0 release introduces cross‑data‑center transfer, cloud synchronization, data aggregation, DDL filtering, schema name conversion, and numerous bug fixes, providing a reliable MySQL‑focused middleware for diverse data replication and migration scenarios.
This article explains the evolution of data transfer in computers—from early CPU‑bound copying to DMA, channel architectures, and I/O processors—then demonstrates how zero‑copy techniques like sendFile and memory‑mapped I/O dramatically reduce CPU usage and improve throughput.
This article explains what the pv (Pipe Viewer) tool is, its functions such as monitoring data flow and rate limiting, how to install it on various Linux distributions, and provides practical command‑line examples for measuring transfer speed and throttling bandwidth.
