BestHub
Sign in
Tagged articles
13 articles
Page 1 of 1
DataFunSummit
DataFunSummit
Jan 16, 2025 · Big Data

Zhihu Big Data Cost‑Reduction Practices: FinOps, Erasure Coding, ZSTD Compression, Spark Auto‑Tuning, and Remote Shuffle Service

This article details Zhihu's comprehensive cost‑reduction and efficiency‑boosting initiatives for its big‑data platform, covering FinOps‑driven financial operations, hybrid‑cloud architecture, cost allocation models, operational monitoring, and technical optimizations such as erasure coding, ZSTD compression, Spark auto‑tuning, and a remote shuffle service.

Big DataCloud Cost ManagementCost Optimization
0 likes · 22 min read
Zhihu Big Data Cost‑Reduction Practices: FinOps, Erasure Coding, ZSTD Compression, Spark Auto‑Tuning, and Remote Shuffle Service
JD Tech
JD Tech
Mar 14, 2024 · Databases

JD ElasticSearch Supports ZSTD Compression: Implementation, Performance Evaluation, and Usage Guide

This article explains how JD ElasticSearch has integrated the high‑performance ZSTD compression algorithm, details the motivations behind its adoption, presents benchmark results comparing it with LZ4 and best_compression, and provides step‑by‑step instructions and code snippets for configuring and using the new jd_zstd codec in Elasticsearch.

Elasticsearchcompressionjava
0 likes · 14 min read
JD ElasticSearch Supports ZSTD Compression: Implementation, Performance Evaluation, and Usage Guide
DaTaobao Tech
DaTaobao Tech
Dec 8, 2023 · Backend Development

ZSTD Compression and GC Optimization in Java Netty Backend

Switching a Java Netty gateway from GZIP to ZSTD compression using zstd‑jni doubled GC time and introduced heap and Netty off‑heap memory leaks, which were resolved by employing the library’s direct off‑heap API with a NoFinalizer compressor, promptly releasing ByteBufs, avoiding finalize(), and adopting jemalloc to reduce fragmentation.

MemoryLeakNettygc
0 likes · 19 min read
ZSTD Compression and GC Optimization in Java Netty Backend
Open Source Linux
Open Source Linux
May 26, 2023 · Operations

Boost Linux Performance with zSwap, zRAM, and Zstandard Compression

This article explains how Linux memory compression techniques such as zSwap, zRAM, and the Zstandard algorithm reduce I/O pressure, extend flash lifespan, and improve overall system performance, while also covering their drawbacks and step‑by‑step activation procedures.

Linuxmemory compressionperformance optimization
0 likes · 6 min read
Boost Linux Performance with zSwap, zRAM, and Zstandard Compression
MaGe Linux Operations
MaGe Linux Operations
May 18, 2023 · Operations

Boost Linux Performance with zSwap, zRAM, and zstd Compression

Memory compression techniques like Linux's zSwap, zRAM, and the zstd algorithm reduce I/O latency and extend RAM capacity by compressing swap pages, offering performance gains while introducing trade‑offs such as CPU overhead and configuration complexity, and this guide explains their principles, advantages, drawbacks, and activation steps.

Linuxmemory compressionsystem performance
0 likes · 6 min read
Boost Linux Performance with zSwap, zRAM, and zstd Compression
Programmer DD
Programmer DD
Feb 8, 2023 · Cloud Native

How Cloud‑Native Pipelines Cut Build Time 3‑5× with Remote Cache

This article explains how introducing a remote cache backed by CFS and Zstandard compression into cloud‑native CI/CD pipelines dramatically reduces build times by 3‑5 times, outlines the implementation steps, tool choices, cache key strategy, eviction policy, and showcases performance gains across Java, Node.js, Go, and GCC builds.

CFSPipelinecaching
0 likes · 10 min read
How Cloud‑Native Pipelines Cut Build Time 3‑5× with Remote Cache
Programmer DD
Programmer DD
Aug 23, 2022 · Cloud Computing

How Switching S3 Compression to Zstd Cut Storage Costs by 30%

AWS switched S3 log compression from gzip to Zstandard, saving roughly 30% of storage space—equivalent to exabytes—by changing the internal storage method, a move confirmed by employee comments and a coinciding 31% price drop in late 2021.

AWSCost OptimizationS3
0 likes · 4 min read
How Switching S3 Compression to Zstd Cut Storage Costs by 30%
Laravel Tech Community
Laravel Tech Community
Jul 5, 2022 · Fundamentals

Linux Kernel 5.19 RC5 Released with New Features and Upcoming Stable Version

Linux kernel version 5.19 has entered its fifth release candidate (RC5), with Linus Torvalds noting a low patch count focused on driver fixes, architecture updates, and filesystem and networking improvements, and introducing support for LoongArch CPUs, Apple NVMe, Zstd-compressed firmware, and FAT32 creation timestamps, with the stable release slated for late July.

LinuxLoongArchNVMe
0 likes · 2 min read
Linux Kernel 5.19 RC5 Released with New Features and Upcoming Stable Version
Meituan Technology Team
Meituan Technology Team
Jan 7, 2021 · Operations

Optimizing Build Package Compression for Faster Deployment

By integrating SIMD‑accelerated ISA‑L and parallel Zstandard (Pzstd) into Meituan’s build‑and‑deployment pipeline, compression time for typical 200 MiB–1 GiB packages dropped from over a minute to about one second—a 90%‑plus speedup that preserves gzip compatibility while dramatically accelerating overall build latency.

DeploymentISA‑Lbuild
0 likes · 21 min read
Optimizing Build Package Compression for Faster Deployment
ITPUB
ITPUB
Apr 3, 2019 · Operations

Why Arch Linux Switched Its Default Package Compression to Zstandard (zstd)

Arch Linux maintainers replaced the single‑threaded xz compression in devtools with Facebook's fast, multithreaded Zstandard algorithm at level ‑18, achieving dramatically higher compression and decompression speeds and more stable parallel processing, which will speed up pacman package installations.

Arch LinuxDevToolscompression
0 likes · 3 min read
Why Arch Linux Switched Its Default Package Compression to Zstandard (zstd)