The article explains why the massive, diverse data generated by the Internet of Things requires specialized databases, outlines key criteria for selecting a suitable database, and reviews several popular options such as InfluxDB, CrateDB, MongoDB, RethinkDB, SQLite, and Cassandra, highlighting their strengths for IoT workloads.
The article evaluates how turning on binlog transaction compression in MySQL 8.0 (both 8.0.19 and 8.0.20) affects throughput and resource usage under a sysbench workload of 120 tables with 240 concurrent threads, showing roughly a 1% performance drop and a similar increase in CPU consumption.
The article introduces the JUST engine, a spatio‑temporal data platform that extends GeoMesa with three new indexes (Z2T, XZ2T, time_range), defines nine common and three specialized data models, provides default indexing strategies, and offers detailed SQL usage guidelines for efficient querying of massive urban datasets.
This article explains the essential role of logging and recovery mechanisms—especially the ARIES algorithm—in ensuring ACID properties, covering physical vs logical logging, optimization techniques like group commit and early lock release, and the three-phase recovery process.
This article explains the concept of sparse indexes in database systems, compares them with dense indexes, and demonstrates their use in Kafka log files and ClickHouse MergeTree tables, highlighting implementation details, lookup procedures, and configuration parameters.
Although Docker has become popular for deploying applications, this article outlines seven critical drawbacks—data safety, performance, networking, statefulness, resource isolation, cloud platform incompatibility, and environment requirements—that make containerizing databases generally unsuitable, while also suggesting scenarios where lightweight or distributed databases may still benefit from containerization.
This article explains how to view MySQL binlog and relaylog events, interpret their formats using commands like SHOW BINARY LOGS, SHOW RELAYLOG EVENTS, hexdump and mysqlbinlog, and describes the underlying replication code paths, parameters, and source‑level handling of binlog transmission and processing.
This article examines common database performance bottlenecks such as I/O and CPU limits, explains the concepts, results, and scenarios of horizontal and vertical sharding—including database and table partitioning—and outlines tools, steps, and challenges for implementing effective sharding solutions.
This article explains how to identify which slave in a MySQL one‑master‑multiple‑slave semi‑synchronous setup is slowing the master’s commit performance by using debug logging and examining the error log for the last ACK server ID.
This article analyzes common database performance bottlenecks such as I/O and CPU limits, explains how horizontal and vertical sharding (including database and table partitioning) can alleviate these issues, introduces practical tools and step‑by‑step implementation guidance, and discusses related challenges like non‑partition key queries and scaling.
This article explains the internal structure of Redis dictionaries, how elements are added, and how rehash-based resizing works, drawing parallels with Java's HashMap to help interviewees answer related questions effectively in technical interviews.
This article explains why SQL queries do not start with SELECT, outlines the typical logical execution order of clauses such as FROM, WHERE, GROUP BY, HAVING, SELECT, ORDER BY, and LIMIT, and discusses how database engines may reorder operations for optimization, with code examples and comparisons to LINQ and pandas.
The article compares traditional relational databases with NoSQL, explains why NoSQL emerged, outlines its four main categories—key‑value, document‑oriented, column‑family, and graph databases—lists popular implementations, their features, and suitable and unsuitable application scenarios.
The article starts with a job‑interview anecdote and then provides a comprehensive overview of NoSQL, its history, definitions, comparison with MySQL, practical scenarios, and detailed examinations of key NoSQL projects such as LevelDB, RocksDB, SSDB, and Pika, highlighting their architectures and use cases.
This article explains the CAP theorem, its trade‑offs, the BASE model, various consistency levels, and how parameters N, W, and R are used to achieve strong or eventual consistency in distributed database systems.
This article demonstrates how to create a test database, export data with mysqldump, and then replay a large SQL dump into a busy MySQL instance while throttling the import speed with the pv tool to keep CPU usage low and avoid disrupting ongoing business operations.
This article examines the fundamental differences between traditional disk‑based DBMS and modern in‑memory databases, covering buffer management, lock versus latch mechanisms, logging and recovery, performance overhead, historical evolution, and architectural innovations that enable high‑performance memory‑resident data processing.
This article explains the concepts, requirements, and practical examples of the first, second, and third normal forms in relational databases, showing how to identify repeated groups, handle composite keys, and eliminate transitive dependencies to achieve a well‑structured schema.
This article explains database concurrency control mechanisms, detailing transaction fundamentals, isolation levels, associated anomalies, and how DB2, MySQL, and Oracle implement locks and isolation to ensure data consistency while balancing performance.
This article provides a comprehensive overview of database technology, covering relational and NoSQL systems, the historical evolution from navigational to self‑driving databases, current market size and growth, leading vendors, and the accelerating shift toward cloud‑based database services.
This article explains Redis 6.0’s built‑in client‑side caching, demonstrates how to install Redis via Docker, configure Lettuce snapshot dependencies, write Java code to enable and test cache synchronization, monitor Redis commands, and outlines current limitations of Lettuce and Spring Boot support.
The article provides a comprehensive analysis of major relational databases—Oracle, DB2, SAP HANA, MySQL, PostgreSQL—and emerging NoSQL solutions, examining their underlying technologies, recent developments, market adoption, and the overall industry landscape shaping the future of data storage.
This article offers a systematic approach to selecting and reading technical books, distinguishing methods for beginners and experienced developers, and illustrates the process with a PostgreSQL case study, emphasizing repeated reading and hands‑on practice as the key to mastering technical material.
This article tests whether MySQL re‑executes an UPDATE statement that does not change any column values, comparing the behavior under binlog_format=ROW and binlog_format=STATEMENT on MySQL 5.7.25 running on CentOS 7.4.
This article reviews B+ trees, introduces log‑structured merge (LSM) trees, compares their strengths and weaknesses, and explains how HBase leverages LSM trees, HFiles, compaction, and Bloom filters to achieve high‑performance storage for write‑intensive workloads.
Redis creator Salvatore "Antirez" Sanfilippo announced he will stop daily coding for the project, sharing his reasons, the history of Redis, the challenges of maintaining a popular open‑source database, and his plans to become a technical advisor while pursuing personal interests.
This article explains the background, competitive landscape, requirements, and architectural design of Tencent CloudBase's cloud database for mini‑programs, detailing its security, ease of use, low‑cost, high‑performance, and flexible solutions, including access control, data safety, elastic scaling, hot migration, and intelligent DBA features.
RedisDesktopManager 2020.1 introduces enhanced Pickle formatting for NumPy and pandas objects, adds the ability to save values to files, and fixes several bugs including dark‑mode and formatter issues, while continuing to support fast, cross‑platform Redis management via SSH tunnels.
Elasticsearch 7.8.0 introduces new aggregation capabilities, composable index template simulation, geo enhancements, SQL date‑time functions, and several breaking changes such as optimized value_count aggregation and mapping restrictions, providing a comprehensive update for search and analytics workloads.
This article provides a concise yet comprehensive overview of the backend development technology stack, covering load balancing, microservice ecosystems, RPC frameworks, service discovery, relational and NoSQL databases, caching strategies, message queues, object storage, and search engines, while highlighting practical configurations and real‑world trade‑offs.
In a June 10 live broadcast, ICBC software development manager Wei Yadong answered 31 audience questions covering the motivations, architecture, storage, high‑availability, migration challenges, and operational practices behind the bank’s distributed database selection and design decisions.
This article explains how databases store data by rows or columns, compares their structures, performance trade‑offs, and use cases, and clarifies the distinction between column‑oriented systems and wide‑column stores such as BigTable and HBase.
This guide explains how SQL analytic (window) functions work, covering the concepts of current row, logical and physical windows, PARTITION, ORDER BY, the differences between ROWS and RANGE, window clause variations, and their advantages and drawbacks.
The 2020 Stack Overflow Developer Survey, involving nearly 65,000 developers, reveals MySQL as the most popular database, highlights Redis, PostgreSQL and Elasticsearch as the most loved, lists DB2 and Oracle as the most feared, and presents salary and career trends for database‑related roles worldwide.
Based on responses from nearly 65,000 developers worldwide, the 2020 Stack Overflow Developer Survey highlights shifting popularity of languages like Rust and TypeScript, rising DevOps salaries, preferred frameworks, database choices, work habits, and demographic insights, offering a comprehensive snapshot of the current software development landscape.
This article provides a comprehensive technical overview of Tencent Cloud's TBase database, covering fundamental database classifications, TBase's cloud‑native architecture, distributed transaction mechanisms, storage and compute separation, security layers, backup strategies, and real‑world deployment cases.
This article explains how Redis uses memory, covering memory fragmentation analysis, object and buffer memory consumption, sub‑process memory overhead, and practical configuration of maxmemory limits and eviction policies to optimize performance and prevent out‑of‑memory failures.
This article explains how HBase uses WAL and MemStore to create HFiles, describes the two compaction types (Minor and Major), and provides detailed recommendations for tuning key compaction-related configuration parameters to improve query performance and reduce HDFS impact.
This roundup covers why ops teams should hire more women, the low female representation in Chinese IT operations, a high‑scoring Python exam, legal cases of source‑code theft and data‑breach, a major cloud procurement in Zhejiang, plus the latest releases of Grafana 7.0 and Redis 6.0.3, and Zoom’s service change in China.
This article compares common database date‑time storage options—string fields, DATETIME, TIMESTAMP, and Unix timestamps—explaining their formats, range limits, timezone handling, performance implications, and provides Java code examples and SQL conversion tips to help developers select the most suitable approach for their systems.
This article categorizes and explains the most important HBase 1.1.2 configuration parameters—covering Region sizing, BlockCache strategies, Memstore thresholds, Compaction behavior, HLog handling, Call Queue tuning, and miscellaneous settings—while offering practical recommendations for optimal production deployment.
This article offers a detailed, experience‑driven comparison of Elasticsearch against its main rivals—Lucene, Solr, relational databases, OpenTSDB, HBase, MongoDB, ClickHouse, and Druid—highlighting where Elasticsearch excels, where it falls short, and practical guidance for choosing the right data solution.
This guide presents ten practical Redis best‑practice tips—including avoiding KEYS *, using SCAN, analyzing command stats, leveraging hashes, setting expirations, choosing eviction policies, handling errors, scaling with clusters, understanding threading limits, and configuring Sentinel for high availability—to help you run Redis efficiently and safely.
The article explains how to approximate the progress of mysqlbinlog when parsing large MySQL binary logs by observing the file handle's read offset, since the tool lacks a built‑in progress indicator.
The article explains how maintaining an optimal number of HBase regions (typically 20‑200 per RegionServer) and appropriate region size, along with careful MemStore and compaction settings, can prevent memory pressure, reduce GC pauses, and enhance overall cluster stability and throughput.
This article explains the three Redis expiration‑key deletion strategies—timed, lazy, and periodic—their advantages and disadvantages, how Redis implements lazy and periodic deletion, and how persistence mechanisms such as RDB, AOF, and replication handle expired keys.
A 2020 survey of 1,113 Node.js developers, conducted by Ecma members and led by Alibaba and Tencent, reveals demographic trends, preferred languages, frameworks, tools, deployment practices, learning habits, and future interests, providing a comprehensive snapshot of the Node.js ecosystem.
This article explains the HBase data‑write process—including WAL logging, MemStore caching, and HFile flushing—identifies three levels of write‑blocking (HFile, MemStore, RegionServer), and provides configuration tweaks to mitigate blocking in production environments.
The article announces that twenty‑seven distinguished professionals from fields such as blockchain, AI, big data, databases, and distributed systems have joined Tencent Cloud’s Valuable Professional program, highlighting their backgrounds, achievements, and commitment to advancing cloud computing and enriching the developer ecosystem.
The article examines why upgrading a three‑shard MongoDB cluster from version 3.2 to 4.0 causes CPU usage to jump from 10 % to 40 %, attributing the increase to more frequent moveChunk operations and the associated deletion workload, and explains the underlying balancer threshold differences between the versions.
This article analyzes common database I/O and CPU bottlenecks, explains horizontal and vertical sharding concepts, presents practical sharding rules, discusses tooling such as Sharding‑Sphere, TDDL and Mycat, and outlines step‑by‑step procedures and common pitfalls for scaling relational databases.
The article reviews a Tencent Cloud TVP closed‑door forum where experts compared MySQL 8.0, PostgreSQL, Redis and the cloud‑native CynosDB, highlighting each product’s technical strengths, ecosystem support, and the broader industry trend toward cloud‑native, multi‑database solutions as the future of data management.
The 2020 JRebel Java technology report reveals that Java 8 dominates usage, Tomcat remains the leading application server, Spring Boot is the preferred framework, IntelliJ IDEA tops IDE preferences, PostgreSQL and Oracle lead databases, Gradle slightly outpaces Maven, Docker is the most common container tool, and microservices continue to shape technology choices.
This article examines common MySQL performance pitfalls—such as large OFFSET limits, implicit type conversion, inefficient UPDATE/DELETE joins, mixed ordering, EXISTS subqueries, and poor condition push‑down—and demonstrates concrete query rewrites that reduce execution times from seconds to milliseconds.
The article explains PostgreSQL 11’s new JIT expression compiler, describes how it was benchmarked using the TPC‑H Q1 query on a 22 GB dataset, presents performance improvements over PostgreSQL 10, and provides details on the testing environment, configuration files, and future expectations.
This article reveals how open‑source software like MySQL powers critical systems—from hospital patient records to massive retail operations—illustrating that a single code error or missing backup can affect tens of millions of people worldwide, underscoring the profound responsibility of open‑source contributors.
This article presents a comprehensive overview of handling spatiotemporal data using Cassandra, covering data types, space‑filling curves, GeoHash encoding, the GeoMesa and GeoTrellis ecosystems, Cassandra storage schemas, and practical Spark integration for large‑scale geospatial analytics.
Redis creates 16 logical databases by default, a design inherited from its early architecture; this article explains the origin of that number, how to modify it via configuration, the proper way to view these databases as namespaces, and the limitations when using Redis in cluster mode.
This article examines the MySQL C API option MYSQL_OPT_READ_TIMEOUT, comparing official documentation that claims a three‑times timeout with empirical tests and source‑code analysis across MySQL 5.7.20, 5.0.96, and later versions, revealing outdated documentation and a historic bug.
In a 2019 OS2ATC talk, Ant Financial’s system department head explained how the company tackles massive data pressure, ultra‑high availability, secure containers, confidential computing, and open‑source initiatives such as OceanBase, Occlum, SOFAMesh, and Kata Containers to drive financial‑grade system software innovation.
This article explains how Redis implements location-based “nearby people” features using GEOADD and GEORADIUS commands, detailing their syntax, underlying geohash algorithm, source code flow, and performance characteristics, while also discussing related options, complexity, and practical usage considerations.
This weekly newsletter summarizes the latest DB‑Engines December ranking, PYPL top‑10 database trends, new features and bug fixes in the DBLE, DTLE, and TXLE middleware projects, and includes community announcements and links to detailed videos and articles.
This article outlines the three generations of databases—SQL, NoSQL, and NewSQL—explaining the limitations of traditional relational systems, the scalability and consistency trade‑offs of NoSQL, and how NewSQL combines distributed architecture with ACID guarantees, while also listing major NewSQL projects.
This article explains how Redis implements location‑based services using GEOADD and GEORADIUS commands, analyzes their source code, describes the underlying geohash algorithm, and evaluates the time‑complexity of nearby‑people queries, providing practical insights for building efficient proximity features.
This article explains iQIYI's multi‑dimensional approach to database selection, details the practical use and optimization of MySQL, TiDB, Redis, Couchbase, and the internally built HiKV, and offers concrete recommendations for choosing the right database in different scenarios.
This article analyzes a use‑case where each user must keep up to 10,000 historical IDs, compares two Redis‑based implementations—using a List and a String—by presenting code samples and benchmark results, and explains why understanding underlying data structures can dramatically improve performance.
This article explains the layout of a DBLE installation, detailing each top‑level directory such as algorithm, bin, conf, pid, lib, logs, version.txt, viewConf, and key configuration files like wrapper.conf, rule.xml, schema.xml, and server.xml, helping users navigate and manage the middleware effectively.
This article describes the challenges of handling billions of travel records in Qunar's MySQL databases, compares open‑source data sync solutions like Databus and Canal, outlines the legacy system’s issues, and presents a refactored architecture that introduces Otter, ES gateway, and improved aggregation to achieve low‑latency, reliable, and scalable data synchronization.
Over eleven years Alipay’s engineers transformed the daunting Double‑11 traffic surges from chaotic outages into a smooth, scalable system through relentless capacity planning, architectural revolutions, rigorous stress testing, and the adoption of the self‑developed OceanBase database, turning “impossible” goals into everyday reality.
This article explains why distributed databases need precise clocks, compares central, logical, and hybrid clock designs, and shows how hybrid logical clocks (HLC) together with two‑phase commit and other transaction techniques enable consistent, high‑throughput distributed transactions.
A GitHub engineer argues that database foreign keys should be avoided because they shift integrity responsibilities to the DB, incur performance overhead, and cause complications during online schema migrations, illustrating the points with parent‑child tables, gh‑ost, and pt‑online‑schema‑change tools.
A recent data platform salon in Beijing gathered five leading experts who shared practical knowledge on data middle platforms, distributed transaction patterns, SQL audit design, Amazon Aurora's architecture, and JD's large‑scale HBase deployment, offering actionable guidance for modern enterprise data engineering.
The article explains that SQL queries are not executed in the order written—SELECT is evaluated fifth after FROM, WHERE, GROUP BY, and HAVING—illustrates this logical sequence with diagrams, discusses alias handling, optimizer reordering, and shows comparable patterns in LINQ, pandas, and dplyr.
MySQL 8.0.18 adds the EXPLAIN ANALYZE feature, which runs a query and reports actual execution times, row counts, and loop iterations, enabling developers to compare estimated costs with real performance and fine‑tune their queries for better efficiency.
This article reviews Alibaba HBase's evolution toward high availability, covering large‑cluster architecture, reliability metrics (MTTF/MTTR), disaster‑recovery strategies such as data replication and traffic switching, performance optimizations for extreme latency requirements, and lessons learned for building resilient distributed database services.
The article analyzes a MySQL replication issue where a low slave_net_timeout combined with a larger MASTER_HEARTBEAT_PERIOD causes frequent zombie dump threads, leading to massive small relay‑log files, and provides detailed explanations, code references, and step‑by‑step remediation procedures.
Debezium is an open‑source distributed platform built on Apache Kafka that turns existing databases into real‑time event streams by capturing row‑level changes via change data capture, offering source and embedded connectors, flexible topic routing, and features such as snapshots, filtering, masking, and monitoring.
This weekly community digest presents the latest DB‑Engines October ranking with MSSQL’s rebound, the Archery v1.7.0 release enhancing MySQL account management, DBLE’s new features, bug fixes and community Q&A, plus DTLE’s performance improvements and upcoming roadmap, all accompanied by relevant links and images.
This article examines the most popular open‑source and commercial database systems in 2019, presents DB‑Engines rankings, explores single versus hybrid strategies, highlights popular SQL‑NoSQL combos, and identifies the most time‑consuming DBA tasks and key performance metrics.
This article explains the principles, characteristics, and performance trade‑offs of the consistent‑hash (ring‑hash) and jumpstringhash sharding algorithms used by DBLE, presents test results on variance, latency and data balance, and concludes why DBLE prefers jumpstringhash.
This article examines how recent hardware advances such as multi‑core processors, large memory, SSDs, NVM, GPUs and FPGAs have reshaped database system design, outlines the stages from pure academic research to productization, and surveys current database products and research directions leveraging these new devices.
This article explains the differences between structured, semi‑structured and unstructured data, walks through the evolution stages of relational database architectures, evaluates their advantages and drawbacks, and then compares major NoSQL families—KV, search, column‑oriented and document stores—to help engineers decide the right storage engine for their workloads.
The article explains the two main types of data processing—OLTP for real‑time transactional workloads and OLAP for analytical data‑warehousing tasks—detailing their distinct characteristics, resource demands, typical features, major vendor products, and a five‑point comparison of their design and usage.
The article explains how Redis 6 introduced multithreaded network I/O to overcome single‑thread bottlenecks, describes the lock‑free read/write flow, provides benchmark configuration and results showing nearly double GET/SET throughput, and outlines upcoming Redis 6.0 features.
Redis achieves exceptional speed by combining a C‑based implementation, pure in‑memory data storage, a single‑threaded event loop, and non‑blocking epoll I/O multiplexing, while supporting rich data structures and advanced features such as transactions, Lua scripting, and clustering.
RedisConf 2019 highlighted the introduction of multithreaded I/O in Redis 6, detailing its architecture, thread handling of network reads/writes, configuration settings, and benchmark results that show roughly double the throughput for GET/SET commands compared to the single‑threaded Redis 5.0.5.
This article surveys the major software development trends for 2019, covering progressive web apps, chatbots, evolving front‑end frameworks, back‑end language choices, database advancements, cloud adoption, AI breakthroughs, and essential development tools, offering guidance on what to learn next.
This document compiles extensive backend interview questions covering topics such as Java concurrency, JVM internals, design patterns, database optimization, distributed systems, Spring Cloud, Dubbo, and system design, along with promotional information for obtaining detailed answer resources.
This article explains what NoSQL databases are, introduces the CAP theorem that underlies their design trade‑offs, outlines typical application scenarios, discusses their limitations, classifies the main NoSQL families, and reviews four widely used systems—Redis, HBase, Cassandra, and MongoDB.
In this interview with MySQL expert Huang Yan, the speaker shares practical, down‑to‑earth techniques for DBAs and developers to explore MySQL’s source code, covering tools, history inspection, test‑driven understanding, DTrace probes, and the mindset of an effective DBA.
This article explains how MySQL performs DDL operations on large tables without long‑lasting locks, covering the evolution from copy‑table and in‑place methods to Fast Index Creation, Online DDL introduced in 5.6, and third‑party tools PT‑OSC and GH‑OST, with detailed process steps and comparisons.
This article explains HBase's compaction mechanism, covering why it is needed, the differences between minor and major compaction, the conditions that trigger compaction, key configuration parameters, thread‑pool handling, compaction policies, and how compaction influences read and write performance in a large‑scale NoSQL database.
This article traces the historical laws that drove computing growth, examines how Redis, MongoDB and Memcached evolved, compares client‑side, proxy and compute‑storage‑separated architectures, evaluates their trade‑offs, and answers common questions about cloud‑based distributed databases.
This article explains common I/O and CPU bottlenecks in databases, compares horizontal and vertical sharding techniques, introduces practical tools, outlines migration steps, and discusses typical challenges such as non‑partition‑key queries, cross‑shard pagination, and scaling.
This presentation by Alibaba multi‑model database expert Zhang Yun explains how Phoenix enables SQL‑style querying over HBase, covering its architecture, query examples, performance considerations, and typical application scenarios.
The article explains why HBase read operations are complex due to its LSM‑Tree storage and multi‑version design, details the step‑by‑step Scan workflow, discusses the reasons for its multi‑request architecture, compares scan performance with Parquet and Kudu, and offers recommendations for large‑scale data scanning.
This article introduces the main business scenarios, system architecture, and core technologies of a time‑series data platform, covering data collection, processing, storage, analysis, and the specific features required for high‑performance, scalable, and reliable time‑series data management.
This article explains the fundamentals of NoSQL databases, the CAP theorem trade‑offs, and why HBase—a column‑oriented, distributed NoSQL system—offers strong consistency, automatic sharding, high availability, and seamless Hadoop integration, while also outlining its ideal scenarios and limitations.
The article explains vertical and horizontal database sharding, compares hash-modulo and range-based partitioning schemes, discusses their advantages and drawbacks such as hotspot issues and migration challenges, and highlights the trade‑offs when scaling large‑scale order data.
This article explains Redis's memory model—including memory statistics, allocation, object structures, internal encodings, and practical optimization techniques—so developers can accurately estimate memory usage, reduce fragmentation, and improve performance of high‑concurrency applications.
