Tagged articles
Data Archiving
25 articles · Page 1 of 1
Facing slow page loads in a legacy WAF reporting system, we dissected a 1000‑line Java method, introduced hourly aggregation, Redis auto‑increment counters, and scheduled synchronization, eliminating costly SQL scans and achieving sub‑second queries on 1.5 million logs, while outlining remaining optimization opportunities.
Data ArchivingJavaRedis
0 likes · 12 min read

This article presents a comprehensive design for a layered storage and unified scheduling platform that archives MySQL historical data, reduces storage costs, ensures high‑performance queries, and enables efficient data analysis through tiered hot, warm, and cold storage using big‑data technologies.
Data ArchivingDorisFlink
0 likes · 13 min read

This article details a closed-loop Data Lifecycle Management (DLM) framework covering six stages—requirement confirmation, model design, development, testing, monitoring, and archiving—with governance practices, toolchains, and a bank case study showing reduced rework, storage costs, and model drift.
Data ArchivingData Modelingcase study
0 likes · 12 min read

Filing 1.0 is a no‑code, heterogeneous data‑archiving platform that unifies MySQL, HBase, Hive, and Elasticsearch, addressing massive order volumes, multi‑domain requirements, and hot‑cold data separation through a star‑shaped architecture, flexible scheduling, and a four‑component archiving engine.
Data ArchivingDistributed ProcessingETL
0 likes · 14 min read

This article explains how to assess MySQL table size through table capacity, disk usage, and instance resources, illustrates the impact of B+‑tree index depth on query speed, and presents three practical solutions—table partitioning, database sharding, and hot‑cold data archiving—to improve performance.
Data ArchivingMySQLPartitioning
0 likes · 15 min read

This article outlines why databases become slow, introduces a four‑layer optimization framework, and presents eight practical solutions—including data reduction, archiving, sharding, caching, and read‑replica strategies—to help engineers systematically diagnose and resolve the majority of performance bottlenecks.
Data ArchivingNoSQLSharding
0 likes · 20 min read

This article presents a comprehensive overview of Baidu Intelligent Cloud's tape‑library solution, detailing tape and tape‑library fundamentals, the Aries cloud storage stack, data and access models, the end‑to‑end data flow, key architectural design choices, implementation details, and a real‑world case study demonstrating large‑scale cold‑data storage, backup, and retrieval performance.
AriesCold DataData Archiving
0 likes · 28 min read

This article explains Baidu Intelligent Cloud’s tape‑library based cold‑data storage architecture, covering tape media basics, the Aries cloud storage system, its modular design, data flow, write and retrieval processes, and a real‑world deployment case that demonstrates cost‑effective petabyte‑scale archival.
AriesCold DataData Archiving
0 likes · 31 min read

The article analyzes why massive tables become a bottleneck in Spring Boot systems, outlines the drawbacks of sharding, and presents a layered solution—data archiving, read‑write separation with caching, heterogeneous source synchronization via Elasticsearch and Canal, and selective sharding—to enable high‑performance real‑time search.
CanalData ArchivingElasticsearch
0 likes · 13 min read

This article explains how to improve Oracle database performance by creating archive tables and triggers, and demonstrates step‑by‑step PHP code for connecting to Oracle, moving old records, and querying archived data.
BackendData ArchivingOracle
0 likes · 5 min read

The article explains how to archive Kafka real‑time data to Hive using either Flink SQL, which quickly creates partitioned ORC tables but requires timezone handling, or Flink DataStream for more complex pipelines, and offers best‑practice guidance on data quality, system complexity, security, and performance.
Data ArchivingDataStreamFlink
0 likes · 15 min read

This article explains how to assess MySQL table size, identify performance issues caused by massive data volumes, and presents three practical solutions—table partitioning, database sharding, and hot‑cold data archiving—along with detailed SQL examples and considerations for implementation.
Data ArchivingMySQLSharding
0 likes · 16 min read

This article explains why large MySQL tables cause slow queries, analyzes table size, disk usage, and instance capacity, and presents three practical solutions—table partitioning, database sharding (horizontal/vertical), and hot‑cold data archiving—helping developers choose the appropriate strategy for their workloads.
Data ArchivingMySQLSharding
0 likes · 18 min read

This guide explains how to use Percona Toolkit's pt-archiver for MySQL data archiving, covering installation, basic and bulk commands, underlying two‑phase‑commit logic, performance trade‑offs of various parameters, and best‑practice tips for safe, efficient archiving and deletion.
Data ArchivingMySQLReplication Lag
0 likes · 19 min read

This article presents a systematic overview of eight practical database optimization techniques—including data reduction, serialization, archiving, intermediate tables, sharding, distributed caching, master‑slave replication, and appropriate storage selection—to help backend engineers diagnose and resolve the majority of performance bottlenecks caused by data volume, high load, and search complexity.
CachingData ArchivingNoSQL
0 likes · 19 min read

This article explains why databases become slow, introduces a four‑layer thinking model, and presents eight practical solutions—including reducing data volume, leveraging space for performance, and selecting the right storage system—to help backend engineers resolve 80‑90% of common performance problems.
Data ArchivingNoSQLPerformance Tuning
0 likes · 19 min read

This article presents a detailed case study of optimizing a massive MySQL table using time‑based partitioning, describing the problem background, partition design, migration scripts, performance benchmarks, and ongoing maintenance procedures for a ten‑year data archive.
Data ArchivingMySQLPartitioning
0 likes · 13 min read

This article explores the business scenario of data archiving in autonomous driving platforms, explains why a serverless architecture was chosen, and details how OpenFunction combined with Dapr, KEDA, and microservices addresses challenges such as diverse storage types, scaling, and observability, providing a flexible, elastic solution.
Data ArchivingKEDAOpenFunction
0 likes · 10 min read

The article explains how to scale MySQL for massive traffic by combining Redis caching, read‑write splitting, data archiving, and sharding techniques, while highlighting their benefits, implementation steps, and potential consistency challenges in modern systems.
Data ArchivingMySQLRead‑Write Splitting
0 likes · 11 min read


DevOpsDec 10, 2020 · Fundamentals
GitHub’s Arctic Code Vault initiative has archived millions of open‑source projects on film stored in a Norwegian Arctic facility, and the company now plans to preserve code for millennia using quartz glass through Microsoft’s Project Silica, highlighting long‑term data preservation strategies for software.
Arctic Code VaultData ArchivingGitHub
0 likes · 4 min read

This guide explains how to safely archive massive MySQL tables by using a custom reimplementation of Percona's pt-archiver that creates a temporary copy, swaps tables via triggers, and moves old data to an archive, reducing disk usage without deleting original rows.
Data ArchivingMySQLSQL
0 likes · 8 min read

The article explains how DNA storage encodes binary data using nucleobases, outlines its massive theoretical density and longevity, describes the required codec, synthesis, and sequencing components, and examines current technical challenges, recent research milestones, and future prospects for commercial adoption.
DNA StorageData ArchivingSynthetic Biology
0 likes · 8 min read

This article explains the components, workflow, advantages, and limitations of physical tape libraries, reviews major tape technologies such as DAT, DLT, LTO, AIT and VXA, and compares them with virtual tape libraries and related backup software integrations.
Data ArchivingLTFStape library
0 likes · 15 min read


ITPUBApr 20, 2016 · Databases
This article examines a MySQL table storing large JSON logistics data, analyzes why its size grows to 130 GB, and presents practical optimizations such as data archiving, JSON field reduction, and schema redesign to keep the table manageable.
Data ArchivingJSONMySQL
0 likes · 8 min read

This article explains why many companies over‑invest in Hadoop, outlines how to evaluate data size, growth, and relevance, and presents practical alternatives such as archiving, data sampling, database sharding, and hiring business‑savvy analysts before committing to a Hadoop deployment.
Big Data AlternativesData ArchivingHadoop
0 likes · 9 min read
