This article chronicles ByteDance's journey from early MySQL‑based databases to a sophisticated, cloud‑native, distributed database platform that now supports over 10 EB of storage, detailing the challenges, architectural milestones, and future directions of its database infrastructure.
This article highlights Ant Group’s recent technical achievements—including green carbon‑reduction scheduling, the AntDTX trusted privacy sandbox for mobile anti‑fraud, the open‑source YinYu privacy‑computing platform, the OceanBase database competition, and the Space5D security parallel‑slice architecture—showcasing their impact on sustainability, security, and open‑source collaboration.
The May 2023 DB-Engines ranking shows Oracle gaining the most points while MySQL and SQL Server plunge, and PostgreSQL and Redis surge dramatically, with detailed score changes, top‑10 lists across database categories, and the methodology behind the popularity metrics.
Serverless, once a niche concept, is rapidly becoming a core paradigm in cloud computing, offering resource decoupling, automatic scaling, and usage‑based billing, while its adoption in databases introduces unique challenges such as persistence, connection handling, and performance, prompting ongoing research and evolving commercial implementations.
This article explains the evolution of cloud service models, defines cloud‑native databases, and presents a detailed maturity model—from basic cloud‑ready deployments to fully serverless data services—helping enterprises evaluate and adopt the most suitable architecture and technology stack for their workloads.
This article chronicles JD Baitiao’s journey from a monolithic MySQL setup through NoSQL and DBRep to a mature ShardingSphere‑based sharding solution, highlighting the technical motivations, architectural decoupling strategies, evaluation criteria, performance comparison, and the operational benefits achieved for a high‑traffic financial service.
This article presents a detailed case study of an IoT device metadata management platform, describing the business scenario, the evolution from a single‑node MySQL solution through sharded MySQL, HBase and Elasticsearch, to a self‑developed distributed storage engine that separates compute and storage, supports LSM, multi‑dimensional indexing, routing keys, and parallel scans to meet massive write‑read throughput and complex query requirements.
This article introduces the concept and practice of Change Data Capture (CDC), explaining why CDC is needed for real‑time analytics, how it works by capturing DML changes, modern approaches using transaction logs, and key considerations for building a production‑ready CDC system.
This article explains how large hotel‑booking platforms such as Airbnb, Booking.com and OYO use a micro‑services architecture, load balancers, master‑slave databases, caching, message queues, Elasticsearch, Cassandra and Hadoop to deliver seamless hotel management, search, booking and view‑booking services at massive scale.
This article chronicles the step‑by‑step evolution of a ride‑hailing order service, detailing how a single SQLServer database was optimized, split by business domain, migrated to MySQL, and finally sharded with a custom client‑side component, while leveraging read/write separation, caching, MQ, and data‑exchange platforms to sustain massive traffic.
A recent poll revealed that MySQL with the InnoDB engine can support up to four billion tables, a limit dictated by the underlying file system rather than MySQL itself, and the official documentation confirms this massive capacity.
Recent developments include a US appellate court affirming that publicly accessible web data can be scraped legally, Google releasing an emergency Chrome 100.0.4896.127 patch for the critical CVE‑2022‑1364 V8 type‑confusion flaw, DB‑Engines’ latest database popularity rankings highlighting Redis’s rise, and Mullvad’s Firefox‑only privacy extension becoming open‑source.
This article explains three common SQL deduplication techniques—using DISTINCT, GROUP BY, and the ROW_NUMBER window function—illustrating their syntax, performance considerations, and example queries for counting unique task identifiers in relational databases.
This article shares experienced DBA insights on optimizing Chinese domestic databases, highlighting why traditional Oracle benchmark tweaks often fail, the limited value of buffer‑hit metrics, the importance of OS I/O tuning, SQL plan baselines, and handling high‑concurrency bottlenecks such as spinlock contention.
This article explains how using IN and NOT IN in SQL queries can lead to poor performance, index misuse, and incorrect results—especially with NULLs—and shows how EXISTS, NOT EXISTS, and JOINs provide more reliable and efficient solutions.
This article explains how MySQL 8.0.27 adds multi‑factor authentication (MFA) via the new authentication_policy system variable, demonstrates configuring the variable, creating users with single or multiple authentication plugins, installing external plugins, and verifies MFA behavior with practical Docker‑based examples.
This article explains Redis's in‑memory database architecture, detailing data locality, various eviction policies such as LRU, LFU, random and TTL, their configuration via maxmemory settings, and provides code examples of the eviction process and memory‑freeing functions.
This article explains how database indexes work by first reviewing binary search trees, then introducing N‑ary trees and B+ trees, illustrating their structure, storage calculations, and search process in InnoDB to show why B+ trees are used for efficient indexing.
This weekly digest summarizes the latest MySQL 8.0 enhancements, replication delay case, user connection memory limits, Oracle‑to‑MySQL synchronization, new SQLE release, DBLE middleware developments, bug fixes, community Q&A, recruitment notices, and upcoming plans, providing concise technical insights for the open‑source community.
This article explains why Redis achieves high throughput—up to 100,000 QPS—by leveraging in‑memory storage, simple data structures, a single‑threaded event loop with I/O multiplexing, and the optional multithreaded I/O model introduced in Redis 6.0, including configuration details and safety considerations.
This guide provides a comprehensive overview of ClickHouse, covering its core use cases, detailed table‑engine choices, cluster design, Zookeeper integration, query and data‑loading best practices, client tools, and key configuration parameters to ensure high performance and reliability in OLAP workloads.
Cloud‑native databases have shifted from simple containerized services to comprehensive, elastic solutions that integrate multiple engines and distributed technologies, offering horizontal scaling, fine‑grained billing, and high‑performance architectures, while enterprises must evaluate vendor perspectives and choose between public‑cloud services or private‑cloud implementations.
To help readers set their 2022 learning goals, this article curates a list of 15 essential technical books covering database fundamentals, MySQL, Redis, Java core, concurrency, JVM, code quality, software craftsmanship, architecture, distributed systems, and microservices, with brief insights and personal recommendations.
This article compiles sixteen categories of learning materials, detailing the number of books for each topic—including Java, databases, caching, messaging, MongoDB, search, Zookeeper, networking, design patterns, data structures and algorithms, distributed systems, cloud‑native microservices, containers, service mesh, DevOps, and operating systems—along with a Baidu link and extraction code for access.
The 2021 year‑end review highlights the rapid rise of Serverless, the accelerated release cadence and security focus of Kubernetes, the deepening of frontend low‑code development, the practical expansion of service mesh, the growing Rust ecosystem, cloud‑native database evolution, the maturation of Software 2.0, and emerging audio‑video innovations driven by the metaverse.
This article compiles the 2021 Qunar Technical Salon infrastructure series, presenting original technical writings on databases, operational practices, and core components, each linked to detailed posts that share real‑world experiences, design guidelines, and performance insights for engineers and practitioners.
The article examines how China’s domestic database market is shifting from legacy commercial systems to cloud‑native solutions like Alibaba Cloud’s PolarDB, highlighting Gartner’s recognition, architectural innovations, open‑source initiatives, and the broader impact on the nation’s digital economy.
2021 marked a pivotal year for databases, with rapid cloud adoption, the rise of HTAP architectures, the decline of Hadoop, and open‑source becoming mainstream, while PingCAP’s TiDB exemplifies DBaaS implementation through cloud‑native design, cost optimization, security, automation, and multi‑region compliance.
Open source databases are freely available software whose source code can be read, modified, and redistributed, and the article explores their licensing models, commercial versus community editions, major vendors' adoption strategies, emerging database types, and the governance and licensing challenges faced by the ecosystem.
The article explains the CAP theorem’s origin, formal proof, detailed definitions of consistency, availability and partition tolerance, its impact on relational and NoSQL databases, various consistency models, and practical trade‑offs for designing distributed data systems.
This article traces the history of data management from early manual and file‑system storage through relational models to modern distributed databases, covering key concepts like CAP theorem, distributed transactions, HTAP, and cloud‑native deployment trends.
The SQLE 1.2201.0-pre2 preview release introduces a SQL Server audit plugin, version display enhancements, and per‑SQL remark support, while providing repository, documentation, and release links, and includes a note that the preview may not guarantee smooth upgrades.
The preview release SQLE 1.2201.0-pre1 introduces audit workflow enhancements, fixes LDAP and MySQL schema issues, provides updated documentation and repository links, and offers several related reading links for the open‑source SQL audit tool aimed at database users and administrators.
This article compiles 15 common interview topics—from sorting linked lists and encryption differences to TCP reliability, I/O models, Hystrix, delayed tasks, HTTPS flow, transaction isolation, index pitfalls, virtual memory, ranking with Redis, distributed locks, zero‑copy techniques, synchronized internals, and Snowflake ID generation—providing concise explanations, code examples, and diagrams for each.
This curated collection gathers essential articles and tutorials covering Java 8‑17 updates, GraalVM performance tricks, Spring Native adoption, Spring Cloud and RSocket alternatives, GraphQL frameworks, observability stacks like Grafana, Prometheus and Loki, IDE enhancements, database fundamentals, and low‑code platform building, providing a comprehensive knowledge base for modern backend developers.
This article explains the concept of eventual consistency in distributed databases, introduces the Hint Switch (HH) queue used by XDB Enterprise to handle failed writes, and discusses how replication factor and node failures affect data consistency, illustrated with practical examples and diagrams.
This article demonstrates how to use the FunTester framework to benchmark MySQL SELECT and UPDATE statements, providing sample scripts, execution results, and insights into performance tuning for typical workloads and discusses observed QPS values.
This article explains the concepts of sequential, causal, and eventual consistency models, illustrating their definitions, guarantees, and practical examples—including process interactions and real‑world scenarios—while comparing their strengths, trade‑offs, and related consistency guarantees such as monotonic reads, writes, and read‑your‑writes.
This weekly community newsletter curates technical articles and updates, covering MySQL index length limitations, new password policies, backup‑induced replication issues, open‑source middleware progress, bug fixes, and upcoming development plans, while inviting reader feedback for further improvements.
The article explains how shifting complex filtering, validation, and join operations from the database to the application layer reduces scarce database resource usage, improves scalability, and enables more flexible resource allocation for modern web applications.
This article outlines ten essential Redis best‑practice tips, covering why to avoid the KEYS * command, using SCAN, interpreting INFO stats, leveraging hashes, setting key expirations, choosing eviction policies, handling errors, scaling with clusters, CPU considerations, and ensuring high availability with Sentinel.
This article provides a detailed, step‑by‑step guide for startups to design and select a complete backend technology stack, covering language choices, components, processes, systemization, and the practical evaluation of tools such as project management, DNS, load balancing, databases, messaging, CI/CD, monitoring, and deployment.
The article explains why the Internet of Things generates massive, diverse data streams that require specialized databases, outlines key selection criteria, describes common IoT data types, and reviews several open‑source databases—InfluxDB, CrateDB, MongoDB, RethinkDB, SQLite, and Cassandra—highlighting their strengths for IoT workloads.
The November 2021 DB-Engines popularity ranking, based on October data, reveals shifts among the top 20 databases, highlights score drops for MySQL, SQL Server and MongoDB, growth for PostgreSQL and MongoDB, and explains the five metrics used to calculate these rankings.
This article explains how to remove duplicate rows in SQL by using DISTINCT, GROUP BY, and the ROW_NUMBER window function, compares their syntax and performance, and provides concrete examples with a Task table and a Test helper table.
This article examines cloud computing’s core principle of using interchangeable machines, compares parallel‑first and single‑storage‑first database architectures, outlines the advantages such as scalability, cost efficiency, and integration, and discusses the weaknesses including maturity gaps, latency, and vendor lock‑in.
The SQLE 1.2111.0-pre1 preview release introduces plugin-based support for arbitrary databases, provides a PostgreSQL plugin example, includes API adjustments, fixes a rule‑passing bug, and offers links to the source repository, documentation, and release information for users and administrators.
This article compares MySQL and HBase across architecture, storage engine, indexing structures (B+ tree vs LSM tree), data access features, and ecosystem integration, highlighting each system's strengths, limitations, and the scenarios where HBase is a suitable complement to MySQL for large‑scale data workloads.
This guide compiles over 300 interview-style questions covering Python fundamentals, network programming, concurrency, databases, caching, frontend frameworks, DevOps tools, and system design, along with code snippets and brief explanations to help candidates prepare for technical interviews.
This article explains Cobar’s sharding middleware and presents an optimization technique for handling ORDER BY and LIMIT queries across multiple databases, detailing the original approach, its inefficiencies with deep pagination, the step‑by‑step improved algorithm, performance analysis, limitations, and practical applicability.
This article details a step‑by‑step performance overhaul for a weekly report service that will exceed 100 million rows, covering indexing, weekly sharding, Redis caching, and data pre‑warming to dramatically cut query latency and reduce database load.
The October DB‑Engines popularity ranking shows the same top‑10 databases as September, highlights score gains for MySQL and PostgreSQL, notes a year‑over‑year decline for the three leading systems, and reveals Snowflake’s rise into the top‑20, all based on five key popularity indicators.
This article provides a comprehensive overview of the Chinese graph database market, covering its origins, government policies, market size estimates, vendor rankings, funding activities, product features, challenges, and future prospects, while referencing numerous industry reports and case studies.
The latest DB‑Engines monthly ranking shows the top nine databases unchanged, highlights a swap between Microsoft Access and Cassandra, notes declines for Microsoft SQL Server and MySQL, and analyzes popularity trends that suggest PostgreSQL and MongoDB may soon surpass SQL Server, illustrated with detailed charts.
The September 2021 DB‑Engines ranking shows Oracle, MySQL and SQL Server plummeting while MongoDB, Snowflake and ClickHouse surge, highlighting a shift toward niche and cloud‑native databases, and the article also recommends ten essential database books for professionals.
Cloud‑native databases address the elasticity, resilience, and high‑availability demands of modern cloud computing by separating compute and storage, leveraging log‑based persistence, multi‑replica consensus, and distributed architectures such as Spanner, Aurora, and TiDB, offering higher performance, lower cost, and better resource utilization.
This article explains the classification of relational and NoSQL databases, analyzes common performance bottlenecks such as query latency, large fields, and write overhead, and presents practical optimization methods including caching, proper indexing, transaction handling, read‑write separation, and sharding for large‑scale systems.
The September 2021 DB‑Engines ranking shows major declines for Oracle, MySQL and SQL Server, rapid gains for Snowflake and MongoDB, and highlights emerging time‑series and key‑value databases like InfluxDB, Redis, and ClickHouse reshaping the database market.
The September 2021 DB‑Engines ranking shows Oracle, MySQL and Microsoft SQL Server retaining the top three spots, while MySQL falls sharply, SQL Server continues a downward trend, and PostgreSQL and MongoDB climb, with detailed score changes and trend charts illustrating shifting database popularity.
This article traces two decades of application data architecture evolution, comparing traditional single‑system LAMP designs with modern multi‑component cloud‑native stacks, and offers practical guidance on scaling, component selection, CDC‑based data derivation, and cloud‑native implementations such as Tablestore.
This article explains the fundamentals of Redis master‑slave architecture, covering why high‑availability replication is needed, the three topology patterns, the three replication modes (continuous, full, partial), the replication backlog buffer, PSYNC command handling, heartbeat mechanisms, and the practical considerations for operators and architects.
This whitepaper explains the concept of in‑memory databases, traces their historical development, outlines core attributes and typical use cases in e‑commerce, live streaming and telecom, compares leading products, and provides technical and management recommendations for hardware and product selection as well as future trends.
The article records a video interview where the candidate answers technical questions on MySQL isolation levels, InnoDB indexing, locking, distributed locks, Redis sharding, PHP‑FPM architecture, RabbitMQ, AOP, Hyperf, and other backend topics, reflecting on the challenges faced during the session.
This article introduces the various Redis eviction strategies—including volatile-ttl, volatile-random, volatile-lru, volatile-lfu, allkeys-random, allkeys-lru, and allkeys-lfu—explains their behavior, shows where they are configured in redis.conf and the initServer source code, and notes the default noeviction policy when memory exceeds maxmemory.
This article outlines a complete backend technology stack for startups, covering language choices, core components, processes, systemization, and detailed selections for project management, DNS, load balancing, CDN, RPC frameworks, service discovery, databases, messaging, logging, monitoring, configuration, deployment, and operational best practices.
This article explains how Redis handles memory exhaustion by setting key expirations with commands like expire and pexpire, describes its lazy and periodic expiration strategies, details the eight configurable eviction policies, and dives into the internal LRU and LFU algorithms used for key eviction.
This article explains Alibaba Cloud's Redis read‑write separation architecture, comparing star and chain replication models, their performance and scalability trade‑offs, and how transparent compatibility, high availability, and high performance are achieved through redis‑proxy, HA monitoring, and optimized binlog replication.
Tencent Cloud's TDSQL‑C database technology was selected for the 2021 SIGMOD conference, showcasing its cloud‑native architecture, massive performance gains, and the award‑winning Spitfire three‑tier buffer manager that leverages DRAM, NVM, and simulated annealing for optimal data migration.
This article explains how MySQL processes ORDER BY clauses, illustrates the execution steps with a full‑field index, discusses the impact of sort_buffer and max_length_for_sort_data, and shows how adding composite and covering indexes can eliminate filesort and improve query performance.
Two Tencent Cloud database papers—Spitfire’s three‑tier buffer manager and a high‑performance strong‑consistency engine for distributed systems—were accepted at ACM SIGMOD 2021, showcasing innovative use of NVM, AI‑driven tuning, and cloud‑native architecture to boost performance and cost efficiency.
The article outlines a range of techniques—including caching, read‑write splitting, vertical partitioning, sharding, cold‑hot data separation, Elasticsearch, NoSQL and NewSQL—to store and retrieve billions of user‑generated records efficiently while maintaining high throughput and low latency.
This document outlines the architectural evolution of the Feed system for To Home Discovery Channel, addressing scalability challenges through database optimization, Elasticsearch integration, and caching strategies to manage billion-scale user data and dynamic content updates.
This article explains the fundamental differences between databases, data warehouses, and data lakes, describes how they complement each other, and introduces the Lake House concept that integrates transactional and analytical workloads using cloud services such as Amazon S3, Redshift Spectrum, and Athena.
This article explains MySQL's three time types, highlights the quirks and automatic defaults of TIMESTAMP—including the 2038 problem, high‑concurrency issues, and unexpected behavior in non‑strict mode—provides practical examples, and offers guidance on configuring server variables or switching to DATETIME for safer handling.
This article explains Redis's memory eviction policies and expiration strategies, detailing how different eviction modes (such as noeviction, allkeys‑lru, volatile‑ttl, allkeys‑lfu) and lazy versus periodic deletion work together to manage limited memory and ensure efficient cache performance.
The article explains how China’s seventh national census, covering 1.41 billion people, was conducted using fully electronic data collection, self‑service mini‑programs, massive cloud‑native infrastructure, and high‑performance databases to achieve real‑time processing and unprecedented scale.
This article presents a curated set of advanced interview questions covering databases, middleware, concurrency, distributed systems, and practical engineering topics, offering both interviewers and candidates a deep dive into essential backend concepts and best‑practice solutions.
The inaugural Techo TVP Developer Summit in Shenzhen gathered over 500 developers to explore the latest trends in databases, distributed systems, big data, and cloud‑native technologies, offering expert analyses, real‑world case studies, and career guidance for data professionals.
This article explains the XA protocol, its two‑phase and three‑phase commit mechanisms, the limitations of traditional two‑phase commit, and how Seata implements an optimized XA mode on MySQL with detailed code examples and architectural diagrams.
This article introduces space-filling curves such as Z‑ordering, Hilbert, and XZ‑Ordering, explaining their mapping algorithms and how they transform multidimensional spatial data into one‑dimensional indices for efficient storage and querying in key‑value databases, while discussing challenges and practical examples.
This article explains the concept, advantages, and inner workings of master‑slave database replication, outlines the key threads and step‑by‑step process, and provides common interview questions with concise answers for candidates preparing for database engineering roles.
This weekly community newsletter compiles the latest technical updates, including MySQL and ProxySQL troubleshooting articles, Redis memory diagnostics, InnoDB performance insights, as well as progress reports and upcoming plans for the open‑source middleware projects DBLE and DTLE.
This article explains how to set Redis's maximum memory usage via configuration files or runtime commands, details all built‑in eviction policies, demonstrates retrieving and changing these policies, and dives into the LRU and LFU algorithms—including Java sample code and Redis's approximate LRU implementation.
Tencent’s TDSQL team introduces a groundbreaking theory that precisely classifies an infinite set of database transaction anomalies, defines their categories via conflict‑graph analysis, and provides an open‑source Linux tool with step‑by‑step compilation instructions for interactive anomaly exploration.
The DB-Engines April 2021 popularity ranking shows the top ten databases unchanged, major score drops for Oracle, MySQL and SQL Server, steady gains for PostgreSQL and MongoDB, and rapid climbs for Snowflake and ClickHouse, with methodology based on five key popularity indicators.
This article explains how Redis handles memory exhaustion by using key expiration commands, describes the three expiration strategies, details the eight maxmemory eviction policies, and dives into the internal LRU and LFU algorithms with their configuration parameters and code examples.
This article details ByteDance's evolution from offline ClickHouse ingestion to a robust real‑time analytics pipeline, covering external transaction handling, risks of direct INSERTs, recommendation and ad‑delivery use cases, Kafka Engine design, multi‑threaded consumption, fault‑tolerance improvements, platform tooling, and future roadmap.
This article details the end‑to‑end process of expanding a high‑traffic ride‑hailing order system on Alibaba Cloud, covering capacity planning, database sharding from 256 to 4096 tables, custom data‑sync middleware, validation, repair, and a gray‑release migration that achieved zero incidents.
This article explains how enterprises can evaluate their needs and compare four popular open‑source relational databases—MySQL, PostgreSQL, MariaDB, and SQLite—covering licensing, performance, language support, strengths, and limitations to guide an informed selection.
A curated list of free Bilibili video courses covering Java, computer networks, operating systems, Redis, RocketMQ, Kafka, and MySQL, each with brief descriptions and direct links for beginners and intermediate learners.
This guide outlines the core JavaScript concepts, asynchronous programming fundamentals, package management, HTTP protocols, web frameworks, databases, caching strategies, template engines, real‑time communication, API clients, and testing tools that form a comprehensive learning roadmap for aspiring Node.js backend developers.
This guide walks through the essential layers of a backend technology stack for startups, covering language choices, core components like DNS, load balancing, CDN, RPC frameworks, databases, messaging, logging, monitoring, configuration, deployment, and operational best‑practice processes.
This article explains the concepts and practices of data modeling, data architecture, master data management, metadata management, and data quality management, describing their roles, key components, and how they support reliable and consistent enterprise information systems.
This article explains how to achieve extensibility in business systems by addressing database scalability through horizontal and vertical sharding, middleware layers, flexible schema designs, and by applying SOA principles to application, interface, and rule‑engine layers for adaptable, high‑performance architectures.
This article introduces Redis as an open‑source, in‑memory data‑structure store, explains its role as a NoSQL database, details its five fundamental data types, common commands, expiration handling, pipelines, and transactions, and shows how it fits into modern scaling architectures.
This article explains the root causes of MySQL master‑slave replication lag—including random replay, high master concurrency, and lock waits—and presents practical solutions such as parallel replication, reducing master load, and reading from the master to ensure data consistency.
Tencent Cloud's native database team announced two breakthrough research papers—Spitfire, a three‑tier buffer manager for DRAM and NVM, and TLBtree, a read/write‑optimized tree index for non‑volatile memory—both accepted at top‑tier conferences SIGMOD and ICDE, highlighting novel memory‑aware storage techniques and collaborative achievements with leading universities.
Redis instances provide 16 default logical databases that act as namespaces rather than separate data stores; this article explains their origin, how to configure the number of databases, proper usage of SELECT and FLUSHALL commands, and why multiple databases are unsupported in Redis Cluster mode.
Time series databases let you retain full historical records, enabling analysis, visualization, machine learning and automation across domains like finance, weather and IoT, and the article explains why they’re essential, how they differ from traditional databases, and how to start using them.
This article introduces ClickHouse as a column‑oriented OLAP database, explains its architectural differences from row‑based systems, details a real‑world migration from MySQL with code examples, showcases performance gains, and shares practical tips and pitfalls for production use.
