BestHub
Sign in
Tagged articles

Zero‑copy

148 articles · Page 2 of 2
Code Ape Tech Column
Code Ape Tech Column
Dec 29, 2021 · Backend Development

Comprehensive Interview Questions and Answers on Backend Development Topics

This article presents a collection of interview questions covering backend fundamentals such as linked‑list sorting, encryption differences, TCP reliability, IO models, Hystrix, delayed tasks, HTTPS flow, transaction isolation, index pitfalls, virtual memory, Redis leaderboards, distributed locks, zero‑copy techniques, Java synchronized, and Snowflake ID generation.

Distributed LocksEncryptionHystrix
0 likes · 30 min read
Comprehensive Interview Questions and Answers on Backend Development Topics
Tencent Cloud Developer
Tencent Cloud Developer
Nov 10, 2021 · Backend Development

Protobuf Shared‑Field Guard for Zero‑Copy User Feature Propagation in Recommendation Systems

The article presents a Guard abstraction that temporarily borrows and returns Protobuf field pointers via set_allocated/release, eliminating costly CopyFrom operations in recommendation pipelines, enabling zero‑copy field sharing across central‑control and recall stages, improving CPU usage and latency while handling safety and rollback.

Performance OptimizationZero‑copyguard pattern
0 likes · 7 min read
Protobuf Shared‑Field Guard for Zero‑Copy User Feature Propagation in Recommendation Systems
Big Data Technology Architecture
Big Data Technology Architecture
Oct 15, 2021 · Backend Development

Key Concepts and Internal Mechanisms of Apache Kafka

This article explains Kafka's internal topics, preferred replicas, partition assignment processes, log directory layout, index files, offset and timestamp lookup, log retention and compaction policies, storage architecture, delayed operations, controller role, legacy consumer design flaws, rebalance workflow, and producer idempotence, providing a comprehensive overview of Kafka's backend architecture.

Consumer OffsetsIdempotenceLog Retention
0 likes · 16 min read
Key Concepts and Internal Mechanisms of Apache Kafka
Liangxu Linux
Liangxu Linux
Sep 11, 2021 · Fundamentals

How DMA and Zero‑Copy Transform Linux I/O Performance

This article explains the costly four‑copy, four‑context‑switch data path in traditional Linux I/O, introduces DMA as a co‑processor that offloads memory transfers, describes zero‑copy techniques such as sendfile, mmap and Direct I/O, and shows how Kafka and MySQL leverage these methods to reduce CPU overhead and improve throughput.

DMADirect I/OKafka
0 likes · 16 min read
How DMA and Zero‑Copy Transform Linux I/O Performance
Open Source Linux
Open Source Linux
Aug 30, 2021 · Big Data

Why Kafka’s Message System Is Essential for High‑Throughput Applications

This article explains why a message system like Kafka is crucial for decoupling services, handling asynchronous workflows such as e‑commerce flash sales, controlling traffic, and achieving high concurrency, high availability, and high performance through sequential disk writes, zero‑copy reads, replication, and careful resource planning.

High concurrencyKafkaMessage Queue
0 likes · 35 min read
Why Kafka’s Message System Is Essential for High‑Throughput Applications
Programmer DD
Programmer DD
Aug 30, 2021 · Big Data

Why Is Kafka So Fast? Unveiling the Secrets Behind Its High Throughput

This article explains how Kafka achieves remarkable speed and massive throughput by using sequential disk I/O, OS page cache, zero‑copy transfers, partitioned log segments with indexes, batch processing, and efficient compression, making it a cornerstone of modern big‑data pipelines.

Big DataKafkaMessage Queue
0 likes · 9 min read
Why Is Kafka So Fast? Unveiling the Secrets Behind Its High Throughput
Java Interview Crash Guide
Java Interview Crash Guide
Jul 19, 2021 · Backend Development

Mastering Zero-Copy in Java: Boost Performance with NIO, Netty, and More

Zero-copy techniques eliminate unnecessary data copying between user and kernel space, dramatically improving I/O performance; this article explains core I/O concepts, explores Java implementations like MappedByteBuffer, DirectByteBuffer, channel-to-channel transfers, and demonstrates Netty’s composite buffers, plus examples from Kafka and RocketMQ.

JavaMemory Mapped FilesNIO
0 likes · 14 min read
Mastering Zero-Copy in Java: Boost Performance with NIO, Netty, and More
Selected Java Interview Questions
Selected Java Interview Questions
Jul 11, 2021 · Big Data

Why Kafka’s I/O Performance Is So High

Kafka’s I/O efficiency stems from sequential data writes, zero‑copy reads using sendfile, batch compression of messages, and a dual‑threaded batch producer that together minimize random disk access and data copying, dramatically speeding up both reading and writing operations.

Batch CompressionIO performanceSequential Write
0 likes · 4 min read
Why Kafka’s I/O Performance Is So High
dbaplus Community
dbaplus Community
Jun 30, 2021 · Backend Development

Unlock Kafka’s Speed: Deep Dive into Performance Secrets and Optimizations

This article provides a comprehensive technical guide to Kafka performance, covering the core bottlenecks of network, disk and complexity, detailing optimization techniques such as concurrency, compression, batching, caching and algorithms, and explaining how Kafka’s sequential write, zero‑copy, page cache, reactor‑based network model, batch handling, partition concurrency, and file structure contribute to high throughput.

JavaKafkaPerformance
0 likes · 17 min read
Unlock Kafka’s Speed: Deep Dive into Performance Secrets and Optimizations
Top Architect
Top Architect
Jun 1, 2021 · Backend Development

Zero‑Copy Mechanisms in Operating Systems and Java

This article explains how zero‑copy techniques such as mmap, sendfile, and splice reduce CPU involvement in data transfer by avoiding memory copies between kernel and user spaces, and shows how Java NIO and Netty implement these mechanisms for high‑performance backend I/O.

Java NIOZero‑copybackend
0 likes · 8 min read
Zero‑Copy Mechanisms in Operating Systems and Java
Intelligent Backend & Architecture
Intelligent Backend & Architecture
Apr 30, 2021 · Backend Development

Why Netty Is the Go-To Framework for High‑Performance Java Backend Development

This article explains how Netty, an asynchronous event‑driven Java NIO framework, simplifies building high‑performance, maintainable network servers and clients by handling I/O, threading, and protocol concerns, and it covers its thread models, zero‑copy techniques, common interview questions, and real‑world use cases such as Dubbo and RocketMQ.

AsynchronousJavaNIO
0 likes · 36 min read
Why Netty Is the Go-To Framework for High‑Performance Java Backend Development
Liangxu Linux
Liangxu Linux
Apr 18, 2021 · Fundamentals

Zero-Copy Explained: How mmap, sendfile, and DMA Boost RocketMQ & Kafka

Zero-copy techniques such as mmap + write, sendfile, and sendfile + DMA scatter/gather reduce CPU copies and context switches by leveraging kernel‑space data transfers and DMA, dramatically improving I/O performance for high‑throughput systems like RocketMQ and Kafka, as explained with detailed process flows and comparisons.

DMAIO performanceZero‑copy
0 likes · 10 min read
Zero-Copy Explained: How mmap, sendfile, and DMA Boost RocketMQ & Kafka
Liangxu Linux
Liangxu Linux
Apr 16, 2021 · Backend Development

How Zero‑Copy and sendfile Boost Server Performance

This article explains the concepts of user space and kernel space, shows how traditional file‑to‑socket transfers involve multiple data copies with read/write system calls, and demonstrates how the Linux sendfile system call implements zero‑copy to reduce copies and system calls for high‑performance servers.

Backend PerformanceLinuxZero‑copy
0 likes · 6 min read
How Zero‑Copy and sendfile Boost Server Performance
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Mar 22, 2021 · Big Data

Kafka Performance Optimization: Sequential Writes, Zero‑Copy, PageCache, and Network Model

This article provides a comprehensive overview of Kafka performance optimization, covering network, disk, and complexity challenges, sequential write techniques, zero‑copy data transfer, page‑cache usage, reactor‑based networking, batch processing, compression, partition concurrency, and file‑structure design to help developers build high‑throughput, low‑latency streaming systems.

Java NIONetworkPerformance
0 likes · 17 min read
Kafka Performance Optimization: Sequential Writes, Zero‑Copy, PageCache, and Network Model
Liangxu Linux
Liangxu Linux
Feb 13, 2021 · Fundamentals

How Linux Zero‑Copy I/O Works: mmap, sendfile, splice & tee Explained

The article explains Linux zero‑copy I/O techniques—mmap, sendfile, splice, and tee—detailing their design principles, execution flows, context‑switch and data‑copy reductions, advantages, limitations, and appropriate use cases, helping developers choose the optimal method for high‑performance file and network transfers.

I/OLinuxTEE
0 likes · 11 min read
How Linux Zero‑Copy I/O Works: mmap, sendfile, splice & tee Explained
Top Architect
Top Architect
Jan 22, 2021 · Backend Development

Introduction to Message Systems and Kafka Architecture

This article explains the purpose of message systems, compares various solutions such as RabbitMQ, Redis, ZeroMQ, ActiveMQ, RocketMQ and Kafka, then details Kafka's design goals, core concepts, architecture, replication, retention policies, zero‑copy transfer, batching, and performance optimizations for high‑throughput distributed messaging.

JavaMessage QueueZero‑copy
0 likes · 22 min read
Introduction to Message Systems and Kafka Architecture
Programmer DD
Programmer DD
Jan 17, 2021 · Backend Development

Mastering Zero-Copy in Java: Boost Performance with NIO, Netty, and Sendfile

This article explains the concept of zero-copy I/O, covering buffer types, virtual memory, mmap+write and sendfile techniques in Java, and demonstrates how Java NIO, Netty, and other frameworks like RocketMQ and Kafka leverage zero-copy to reduce data copying and improve performance.

PerformanceZero‑copymmap
0 likes · 14 min read
Mastering Zero-Copy in Java: Boost Performance with NIO, Netty, and Sendfile
macrozheng
macrozheng
Dec 15, 2020 · Big Data

How Kafka Achieves Million‑TPS Through Sequential I/O, MMAP, and Zero‑Copy

Kafka can sustain millions of transactions per second by writing data sequentially to disk, leveraging memory‑mapped files, employing zero‑copy DMA transfers, and batching messages, each technique reducing I/O overhead and CPU involvement, which together enable its high‑throughput performance in big‑data pipelines.

Big DataKafkaSequential I/O
0 likes · 11 min read
How Kafka Achieves Million‑TPS Through Sequential I/O, MMAP, and Zero‑Copy
Programmer DD
Programmer DD
Dec 1, 2020 · Backend Development

Mastering Zero-Copy in Java: Boost I/O Performance with NIO, Netty & Kafka

Zero-copy techniques eliminate unnecessary data copying between user and kernel space, dramatically improving I/O performance; this article explains core concepts, Java NIO’s MappedByteBuffer and DirectByteBuffer, channel-to-channel transfers, Netty’s composite buffers, and how frameworks like Kafka and RocketMQ leverage zero-copy.

JavaNIONetty
0 likes · 12 min read
Mastering Zero-Copy in Java: Boost I/O Performance with NIO, Netty & Kafka
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 29, 2020 · Fundamentals

Zero-Copy Data Transfer Mechanism: Principles, Implementations, and Applications in Java, Kafka, and Spark

This article explains the zero‑copy data transfer technique, compares it with traditional read/write approaches, shows Java NIO code examples, and discusses its use in high‑performance systems such as Kafka and Spark, highlighting the reductions in context switches and memory copies.

Data TransferJava NIOKafka
0 likes · 16 min read
Zero-Copy Data Transfer Mechanism: Principles, Implementations, and Applications in Java, Kafka, and Spark
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Jul 31, 2020 · Backend Development

Why Kafka Is Fast: Partition Parallelism, Sequential Disk Writes, Page Cache, Zero‑Copy, Batching and Compression

The article explains how Kafka achieves high throughput by using partition‑level parallelism, sequential disk writes with segment files, extensive use of the OS page cache, zero‑copy data paths, request batching and optional compression, while also discussing the underlying disk I/O principles.

KafkaZero‑copybackend
0 likes · 14 min read
Why Kafka Is Fast: Partition Parallelism, Sequential Disk Writes, Page Cache, Zero‑Copy, Batching and Compression
Architect
Architect
Jul 26, 2020 · Backend Development

Understanding Zero‑Copy in Java: I/O Concepts, mmap, Sendfile, and Netty

This article explains the zero‑copy technique in Java, covering basic I/O concepts, buffer management, mmap + write and Sendfile methods, and how frameworks like NIO, Netty, Kafka, and RocketMQ use these mechanisms to eliminate data copies and improve performance.

JavaNIONetty
0 likes · 11 min read
Understanding Zero‑Copy in Java: I/O Concepts, mmap, Sendfile, and Netty
ITPUB
ITPUB
Jul 23, 2020 · Fundamentals

How Zero‑Copy Techniques Slash Data Transfer Overhead in Linux

This article explains why traditional read/write file serving incurs multiple data copies and context switches, then introduces mmap, sendfile, and splice as zero‑copy methods that reduce CPU load, discuss their usage, limitations, and practical pitfalls for high‑performance Linux servers.

I/O optimizationLinuxZero‑copy
0 likes · 11 min read
How Zero‑Copy Techniques Slash Data Transfer Overhead in Linux
Programmer DD
Programmer DD
Jul 13, 2020 · Backend Development

Unlocking High-Performance I/O: Java Zero-Copy Techniques Explained

This article explains the principles of zero‑copy I/O, covering buffer concepts, virtual memory, mmap + write and sendfile methods, and demonstrates Java implementations using MappedByteBuffer, DirectByteBuffer, channel‑to‑channel transfers, as well as Netty’s CompositeChannelBuffer for efficient data handling.

JavaNIONetty
0 likes · 15 min read
Unlocking High-Performance I/O: Java Zero-Copy Techniques Explained
Architecture Digest
Architecture Digest
Apr 18, 2020 · Backend Development

Understanding Zero-Copy in Java: Concepts, Implementations, and Netty

This article explains the fundamentals of zero‑copy I/O, covering buffer and virtual memory concepts, Java NIO mechanisms such as MappedByteBuffer and DirectByteBuffer, channel‑to‑channel transfers, and how Netty implements zero‑copy with composite buffers, providing code examples and diagrams for each technique.

JavaNIONetty
0 likes · 14 min read
Understanding Zero-Copy in Java: Concepts, Implementations, and Netty
Java Captain
Java Captain
Apr 7, 2020 · Backend Development

Zero‑Copy Techniques in Java: I/O Concepts, mmap, Sendfile, and Netty

This article explains the zero‑copy principle, describes core I/O concepts such as buffers and virtual memory, and details Java implementations including mmap + write, Sendfile, MappedByteBuffer, DirectByteBuffer, channel‑to‑channel transfer, and Netty’s composite buffers for high‑performance data transfer.

JavaNIONetty
0 likes · 12 min read
Zero‑Copy Techniques in Java: I/O Concepts, mmap, Sendfile, and Netty
Programmer DD
Programmer DD
Mar 28, 2020 · Backend Development

Why Is Kafka So Fast? Uncover the 11 Performance Secrets

Kafka achieves its remarkable speed by combining sequential I/O, batch processing, compression, zero‑copy, careful client‑side work, and a design that avoids costly fsync and garbage collection, while maintaining durability, ordering, and at‑least‑once delivery, making it a high‑throughput, low‑latency event streaming platform.

Batch ProcessingKafkaMessage Queue
0 likes · 15 min read
Why Is Kafka So Fast? Uncover the 11 Performance Secrets
Tencent Cloud Developer
Tencent Cloud Developer
Dec 18, 2019 · Mobile Development

Zero‑Copy Texture Sharing in Flutter: Using CVPixelBuffer for Native‑GPU Integration

This article explains how to bypass the costly GPU‑CPU‑GPU copy when using Flutter external textures by leveraging CVPixelBuffer's shared memory mechanism, detailing the underlying engine changes, registration and rendering steps, code implementation, and a demo that achieves smooth 60 FPS rendering.

CVPixelBufferExternal TextureFlutter
0 likes · 9 min read
Zero‑Copy Texture Sharing in Flutter: Using CVPixelBuffer for Native‑GPU Integration
Efficient Ops
Efficient Ops
Jul 31, 2019 · Fundamentals

Unlock Faster Linux I/O: Master Zero‑Copy Techniques and Boost Performance

Zero‑copy techniques in Linux eliminate unnecessary data copying between user and kernel space, dramatically reducing CPU overhead and boosting throughput; this article explains why zero‑copy is needed, outlines several methods—including user‑space I/O, mmap, sendfile, DMA‑assisted sendfile, splice, copy‑on‑write, buffer sharing, and netmap—plus their trade‑offs.

I/O optimizationLinuxZero‑copy
0 likes · 13 min read
Unlock Faster Linux I/O: Master Zero‑Copy Techniques and Boost Performance
Big Data Technology & Architecture
Big Data Technology & Architecture
May 14, 2019 · Fundamentals

Zero‑Copy Data Transfer: Principles, Mechanisms, and Applications in Kafka and Spark

This article explains the traditional copy‑based data transmission process, introduces the zero‑copy technique—including basic sendfile(), scatter/gather DMA and mmap support—shows how it reduces context switches and copies, and demonstrates its practical use in Kafka and Spark for high‑throughput workloads.

Data TransferJava NIOPerformance
0 likes · 12 min read
Zero‑Copy Data Transfer: Principles, Mechanisms, and Applications in Kafka and Spark
Meituan Technology Team
Meituan Technology Team
Dec 20, 2018 · Backend Development

Design and Performance Optimization of LruCache in Meituan DSP System

Meituan’s DSP system boosted high‑QPS ad serving performance by layering an LRU cache in front of Redis, then adding time‑based eviction, sharding the cache into HashLruCache instances to cut lock contention, and employing a zero‑copy, reference‑counted design, ultimately cutting average latency to about 20 % of the original and similarly reducing 99.9th‑percentile delays.

Cache OptimizationHashLruCacheLRUCache
0 likes · 15 min read
Design and Performance Optimization of LruCache in Meituan DSP System
Programmer DD
Programmer DD
Sep 8, 2018 · Fundamentals

Understanding Linux User/Kernel Space and I/O Models: From Blocking to Zero‑Copy

This article explains Linux's separation of user and kernel address spaces, details the five classic network I/O models—blocking, non‑blocking, I/O multiplexing, signal‑driven, and asynchronous—and introduces zero‑copy techniques such as mmap, sendfile and splice to reduce data copying overhead.

I/O ModelsKernel SpaceLinux
0 likes · 20 min read
Understanding Linux User/Kernel Space and I/O Models: From Blocking to Zero‑Copy
Architects' Tech Alliance
Architects' Tech Alliance
Jul 9, 2017 · Fundamentals

An Introduction to RDMA: Principles, Operation, and Integration with TCP/Ethernet

This article explains the growing need for more efficient data‑center networking, introduces Remote Direct Memory Access (RDMA) technology, describes its working principles, operation types, and how it can be layered over TCP/Ethernet to reduce latency and CPU overhead in high‑performance environments.

Data CenterHigh-performance computingRDMA
0 likes · 14 min read
An Introduction to RDMA: Principles, Operation, and Integration with TCP/Ethernet