This comprehensive guide explains what MyBatis is, compares it with Hibernate, details its core components, configuration, dynamic SQL, caching, plugins, pagination, and practical usage patterns, providing Java developers with clear examples, code snippets, and visual diagrams to master the framework efficiently.
The article explains how Go's database/sql pool retains connections until Rows, Stmt, or Tx objects are explicitly closed or a timeout context is cancelled, demonstrates experiments showing blocked goroutines when resources aren't released, and offers practical guidelines—using QueryContext with timeouts, deferring Close/Cancel, and always committing or rolling back transactions—to prevent pool exhaustion and improve MySQL reliability.
This article explains the background, benefits, and measurement criteria of Kubernetes Horizontal‑Pod‑Autoscaler (HPA), describes the creation of metric tables and SQL queries for collecting scaling events and CPU usage, and presents a Python‑based workflow that aggregates the data, stores daily reports, validates results, and sends automated email summaries.
This article explains how to use MyBatis dynamic SQL tags—including if, where, set, choose, foreach, sql fragments, and bind—to build flexible queries and updates, with detailed code examples and tips for handling null parameters and avoiding syntax errors.
This article examines a MyBatis bug present in versions prior to 3.4.5 by reproducing the issue, walking through the complete query execution flow from configuration parsing to result handling, identifying the root cause in foreach variable binding, and presenting official fixes and practical work‑arounds.
This guide explains ClickHouse fundamentals, OLAP versus OLTP concepts, columnar storage benefits, core performance techniques, the MergeTree family and its indexing, specialized table engines, installation on Linux, Docker deployment, and integration with HDFS, MySQL, and Kafka for modern analytical workloads.
This article explains why MySQL upgrades can cause syntax, semantic and performance incompatibilities, introduces the Percona Toolkit's pt‑upgrade tool, details its detection logic and common usage patterns, provides a full demo with real query results, and lists essential parameters and precautions for safely migrating from MySQL 5.7 to 8.0.
This article compares the open‑source monitoring systems Zabbix and Prometheus, detailing their architectures, component roles, strengths, and weaknesses, then describes how to integrate Zabbix data into Prometheus using a custom super_exporter and visualise the combined metrics with Grafana.
This guide demonstrates how to construct flexible MyBatis dynamic SQL statements using tags such as if, where, set, choose, foreach, sql fragments, and bind, showing practical examples for conditional queries, updates, and reusable SQL components to streamline backend data access.
This article explains MySQL index fundamentals, covering clustered and secondary indexes, best‑practice design rules such as using auto‑increment primary keys, avoiding large or low‑cardinality columns, respecting the left‑most prefix, and common scenarios that cause index loss, with practical SQL examples.
This article explains MySQL's internal execution process—including the connector, cache, parser, optimizer, and executor—describes privilege tables and verification, discusses query caching removal, outlines logical SQL execution order, and provides practical tips for ordering WHERE conditions to improve performance.
This guide walks you through installing DataGrip, configuring database connections, customizing the IDE, and leveraging its powerful SQL editing, navigation, and data import/export features, providing a comprehensive overview for developers seeking a streamlined database client experience.
This article introduces MyBatis-Mate, an official MyBatis-Plus extension that offers enterprise‑grade capabilities such as dictionary binding, field encryption, data masking, dynamic DDL generation, multi‑datasource sharding, data‑scope filtering, and SQL performance logging, complete with Maven dependency setup and code examples.
This article presents ten practical techniques—ranging from avoiding chat‑tool SQL transmission to using logical deletes, backups, and temporary tables—to help developers and DBAs safely execute and modify data in production environments while minimizing accidental data loss.
This guide explains three SQL techniques—DISTINCT, GROUP BY, and the ROW_NUMBER window function—for deduplicating records, compares their behavior and performance, and provides concrete query examples with a sample task table and additional test cases.
This article demonstrates practical techniques for data masking, covering MySQL string functions for phone and ID masking, a Java library for various sensitive fields, and a MyBatis‑Mate extension that applies custom and built‑in masking strategies through annotations and REST endpoints.
This article demonstrates how to mask sensitive phone numbers and ID cards directly in MySQL, introduces a Java masking library, and provides a complete MyBatis‑Mate extension with custom strategies, configuration files, and sample Spring Boot code to protect personal data in applications.
Zuoyebang’s data team replaced fragmented, slow query solutions with Apache Doris, building a unified real‑time data warehouse that dramatically cut query latency from hours to seconds, streamlined data modeling, and improved reliability across diverse business scenarios, while integrating with Flink, Kafka, and ES via a unified API.
This article examines ClickHouse's join mechanisms, detailing the limitations of standard joins, the advantages of Global joins, and optimization strategies such as hash and merge joins, subquery filtering, and memory considerations, illustrated with SQL examples and source‑code analysis.
This article walks through the complete lifecycle of a MySQL SELECT statement, from establishing the TCP connection and authentication, through query cache, parsing, optimization, and execution, detailing how the server and storage engine collaborate and explaining concepts like index usage, connection handling, and memory management.
This article provides a comprehensive overview of MySQL indexes, explaining their underlying data structures, classification methods, practical usage scenarios, and optimization strategies, while illustrating concepts with diagrams and SQL examples to help developers design efficient query performance.
This article compiles frequently asked questions and detailed solutions covering Apache Doris operational problems, data handling errors, and SQL query issues, providing step‑by‑step guidance, configuration tips, and command examples to help administrators troubleshoot and maintain a stable Doris cluster.
This article explains MySQL index fundamentals, covering B+Tree structures, various index types, creation syntax, and practical optimization techniques such as prefix, covering, and composite indexes, while illustrating how to avoid common pitfalls like index misuse and performance‑degrading scenarios.
This article explains how classic SQL execution engines use the volcano model and expression trees, discusses their performance drawbacks, introduces vectorized execution to reduce overhead, and describes Roaring Bitmap compression methods with container types for efficient storage and processing of integer sets.
This article provides a comprehensive overview of Doris's architecture—including its FE and BE components, metadata management, data organization, execution planning—and details its major features such as adaptive join aggregation, vectorized execution, materialized views, and Elasticsearch integration, supplemented with example DDL and query code.
The article examines the inherent trade‑offs of data lakes—maintaining raw data, enabling efficient computation, and keeping costs low—explains why traditional data‑warehouse approaches fall short, and introduces the open‑source SPL engine that provides multi‑source, file‑based, high‑performance analytics to overcome these limitations.
This article explains how MySQL's EXPLAIN command reveals execution plans through twelve key fields, demonstrates practical analysis of a slow query on a large table, and provides actionable tips such as adding or forcing indexes to dramatically improve query performance.
This article provides a comprehensive overview of Oracle's database products, explaining why they matter, detailing the physical and logical storage structures, instance processes and memory areas, and tracing the evolution from early versions through RAC, multitenancy, and the latest 18c features.
This article provides a comprehensive guide to Apache Doris, covering how to create databases and tables with single and composite partitions, import data via broker and routine loads, understand its aggregate, uniq, and duplicate data models, and leverage rollup and prefix index features for optimized querying.
This article explains how to create a custom MyBatis Interceptor using Java dynamic proxies, detailing the Interceptor interface, the plugin generation process, the wrap and invoke methods, and common use‑cases such as pagination, SQL analysis, encryption, and performance monitoring.
This article translates and expands on a HikariCP wiki post, explaining why a small database connection pool often yields better performance than a large one, presenting benchmark data, a practical sizing formula, and guidance for tuning pools in various environments.
Apache Kylin leverages pre‑computed OLAP cubes on Hadoop/Spark/Flink to deliver sub‑second query responses for massive datasets, detailing its architecture, integration with BI platforms, user security, cube building, monitoring, and storage using HBase, illustrating how it overcomes big‑data analytical challenges.
This article explains ClickHouse's single‑node and distributed JOIN mechanisms, compares ordinary, GLOBAL, Broadcast, Shuffle and Colocate JOINs, illustrates execution flows with code examples, and provides practical recommendations to reduce join size, avoid query amplification, and leverage data pre‑distribution for optimal performance.
This article explains the complete MySQL query processing pipeline, covering the connector, authentication, privilege checks, caching, parser, optimizer, executor, and the detailed SQL execution order with examples of each stage, helping readers understand and optimize their SQL statements.
This article explains how to identify slow SQL statements, analyze execution plans with EXPLAIN, use profiling and tracing tools, and apply specific optimization techniques such as index tuning, query rewriting, and alternative storage solutions across a range of real‑world cases.
This article walks through the challenges of inserting massive data sets with MyBatis, explains the inner workings of batch execution, highlights common mistakes with commit, clearCache and flushStatements, and presents progressively refined code examples—including Oracle-specific optimizations—to dramatically improve performance.
This article details Bilibili's end‑to‑end Presto on Hadoop architecture, covering the multi‑engine SQL stack, dispatcher routing, cluster scale, stability enhancements like coordinator HA and real‑time punish, query limits, Hive UDF compatibility, insert‑overwrite support, Alluxio caching, multi‑datacenter routing, query result caching, Raptorx local cache, JDK upgrades, dynamic filtering, and future roadmap, illustrating how these innovations boosted query throughput and reduced latency.
This article walks readers through MyBatis batch processing from basics to advanced, highlighting common mistakes, explaining the roles of commit, clearCache, and flushStatements, and presenting three progressively refined code versions plus Oracle-specific optimizations to dramatically improve bulk insert performance.
This article details JD Logistics' journey from fragmented, massive‑scale data to a unified, real‑time analytics platform, covering business needs, pain points, tool evaluation, a new Doris‑based architecture, table management, data import procedures, automation scripts, and future roadmap for data engineering.
This article explains why using COUNT(*) for simple existence queries is unnecessary and demonstrates how replacing it with SELECT 1 LIMIT 1 in SQL, together with a Java DAO method that checks for non‑null results, can significantly improve performance and reduce index complexity.
Zuoyebang’s big‑data team shares how they evolved from SparkStreaming to a Flink‑SQL‑centric real‑time platform, detailing three development stages, challenges in DAG optimization, Redis‑based table design, and platform features for unified deployment, ease of use, and operational governance.
This article introduces a Spark offline development framework that separates configuration from code, supports SQL and Java applications, and provides fast, automated data backtracking with reduced environment preparation time, lower failure rates, and significant performance gains for large‑scale data warehouses.
This article explains how MySQL InnoDB implements full‑text search using inverted indexes, covers creation and usage of full‑text indexes, demonstrates natural language, boolean, and query‑expansion modes, and shows how to delete full‑text indexes with practical SQL examples.
This article explains what SQL window functions are, shows their syntax, and provides step‑by‑step basic and advanced examples—including how to rank rows within groups and reset rankings when key columns change—complete with runnable SQL scripts and result screenshots.
This article explains why using SELECT 1 with LIMIT 1 is more efficient than COUNT(*) for existence checks in SQL, shows the typical count‑based approach, provides optimized SQL and Java examples, and discusses the performance benefits and reduced indexing requirements.
MySQL creates two kinds of temporary tables—external and internal—based on CREATE TEMPORARY TABLE statements or internal query processing, and uses the MEMORY engine by default unless large objects or the big_tables variable force MyISAM or InnoDB, with detailed guidance on scenarios, storage engine selection, conversion thresholds, indexing, and performance tuning.
This article compares Elasticsearch and ClickHouse in architecture, query capabilities, and performance for log search workloads, presenting Docker‑compose setups, data ingestion pipelines, sample queries, and benchmark results that show ClickHouse generally outperforms Elasticsearch, especially in aggregation scenarios.
In MySQL, an UPDATE that modifies no actual data returns an affected‑rows count of zero, making it unreliable for business logic that distinguishes between unchanged rows and non‑existent records, so developers should avoid using this count for critical decisions.
Facing a sudden MySQL table lock? This guide walks through real‑world troubleshooting steps—from checking table usage and process lists to inspecting InnoDB transaction and lock tables, identifying lock‑wait relationships, and safely killing offending sessions, while also explaining lock types and common metadata‑lock scenarios.
This guide explains what MySQL binlog is, how to enable it in a Docker container, common binlog commands for inspection and maintenance, and demonstrates a step‑by‑step data recovery scenario using mysqlbinlog to restore a mistakenly dropped database.
This article recounts a production incident caused by a MySQL query with a huge OFFSET, explains why large offsets lead to full‑table scans and severe latency, and presents three practical solutions—including index‑covering subqueries, start‑position redefinition, and offset throttling—along with data‑generation scripts and performance test results.
This article explains Flink's window table-valued functions (TVF), shows how they replace the old grouped‑window syntax with concrete SQL examples, describes the physical planning rules, introduces sliced windows for state efficiency, and presents a small incremental‑output improvement for cumulative windows.
The article asks how many tables can be created in a single MySQL database using the InnoDB storage engine, prompting readers to provide the correct limit without consulting external sources and discuss the theoretical maximum based on MySQL specifications.
Facing server‑hardware constraints, Bilibili’s data platform replaced wasteful full‑partition tables with a zipper‑table approach—preserving change history while cutting storage from petabytes to terabytes—and complemented it with Hudi + Flink CDC for near‑real‑time updates, dramatically lowering I/O, compute usage and latency.
This article explains two techniques for creating an org.apache.ibatis.jdbc.SQL object in MyBatis—using an anonymous inner class with an initializer block and using method‑chaining builder style—provides code examples, explains the underlying Java syntax, and summarizes their differences.
This article outlines several frequent MySQL query pitfalls—including missing table aliases in multi‑table operations, unsupported FULL JOIN syntax, unexpected REPLACE INTO side effects, auto‑increment edge cases, data‑type mismatches, nondeterministic functions, and warning‑driven query behavior—providing examples and best‑practice recommendations to prevent data loss and performance issues.
This article presents eight common MySQL error scenarios—including forgotten passwords, password policy violations, case‑sensitivity issues, service startup failures, export/import restrictions, connection limits, binary log overflow, and primary‑key replication errors—along with detailed troubleshooting steps and configuration commands to resolve each problem.
This article explains how to retrieve sales statistics for multiple business units and their associated products in MySQL by comparing rows, discusses alternative approaches such as loop queries, OR concatenation, and mixed filtering, and demonstrates why row‑by‑row comparison is the most suitable solution.
This article examines frequent SQL anti‑patterns—such as inefficient LIMIT usage, implicit type conversion, suboptimal UPDATE/DELETE joins, mixed ordering, unnecessary EXISTS clauses, condition push‑down failures, early limiting, and intermediate result handling—and provides rewritten queries that dramatically improve performance.
The article explains Propensity Score Matching as a causal inference method, detailing treatment effect concepts, required assumptions, score estimation, various matching algorithms, SQL implementation, quality assessment metrics, and how to estimate ATT using Difference-in-Differences, while outlining workflow steps, trade-offs, and alternatives.
This article explains how to replace inefficient recursive MySQL tree queries with the nested‑set (preorder traversal) technique, covering descendant retrieval, subtree counting, leaf detection, insertion, deletion, direct‑child lookup, ancestor path queries, and a JavaScript example for building a tree structure.
This guide explains how to set up the Flink SQL client to work with Apache Iceberg, covering Scala version requirements, downloading and deploying Iceberg jars, configuring Hive and HDFS catalogs, creating databases and tables, performing insert and overwrite operations, and querying data in both batch and streaming modes.
This article explains how MySQL stores indexes using B+ trees, how to evaluate index selectivity and prefix length, the importance of the leftmost principle for composite indexes, and practical techniques such as covering indexes, index condition pushdown, and avoiding common pitfalls that render indexes ineffective, all illustrated with real‑world query examples and performance measurements.
The article explains Presto’s join processing by detailing the business need to limit multi‑table joins, then describing nested‑loop, sort‑merge, and hash join algorithms with Java examples, and finally showing how the Volcano model, columnar pages, and planner integration enable scalable, efficient OLAP join execution.
This article explains typical MySQL index failure cases such as left‑most matching truncation, implicit conversion, IN + ORDER BY, range queries, leading wildcard, OR conditions, function usage, NOT IN/!=, low selectivity, and index fragmentation, and provides practical optimization strategies including index redesign, covering indexes, and pagination improvements.
This article explains three common SQL deduplication techniques—using DISTINCT, GROUP BY, and the ROW_NUMBER window function—illustrating their syntax, performance considerations, and example queries for counting unique task identifiers in relational databases.
This article explains how MySQL implements indexes using B+ trees in InnoDB and MyISAM, compares primary and secondary indexes, demonstrates performance gains from proper indexing on large tables, and provides practical commands for creating, viewing, and dropping indexes.
This article presents a collection of intermediate to advanced SQL interview problems covering grouping, window functions, sessionization, ranking, and join techniques, each accompanied by detailed solution queries and explanations to help data engineers prepare for technical interviews.
The article explains why using IN and NOT IN in SQL can lead to poor performance, incorrect results with NULLs, and index misuse, and demonstrates how EXISTS, NOT EXISTS, and JOIN provide safer and faster alternatives for database queries.
This article details Bilibili's end‑to‑end Presto‑based SQL‑on‑Hadoop architecture, covering overall system components, query routing, Presto feature set, extensive stability and availability enhancements, performance boosts through caching and multi‑datacenter deployment, and outlines future development plans.
This article explains how to replace parent‑id based tree storage with the nested set model, showing SQL queries to retrieve descendants, count them, detect leaf nodes, and perform insert, delete, and other operations, plus a JavaScript example for rebuilding the hierarchy.
This article explains how using IN and NOT IN in SQL queries can lead to poor performance, index misuse, and incorrect results—especially with NULLs—and shows how EXISTS, NOT EXISTS, and JOINs provide more reliable and efficient solutions.
This article provides a thorough guide to SQL fundamentals—including SELECT, FROM, WHERE, GROUP BY, HAVING, ORDER BY, LIMIT, aggregate functions, joins, set operations, and Oracle-specific objects—followed by a collection of common interview questions with detailed solutions, top‑N queries, and row‑to‑column transformations, while also containing promotional notes for related resources.
This article explains how to use the nested set (left‑right) model to efficiently query, count, and manipulate hierarchical department data in MySQL, providing SQL examples for retrieving descendants, ancestors, leaf detection, as well as insert, delete, and JavaScript tree‑building techniques.
Bilibili’s Presto on Hadoop combines a multi‑engine offline platform with Kubernetes‑managed YARN scheduling, Ranger security, and a custom dispatcher, scaling to over 400 nodes handling 160 k daily queries on 10 PB, while adding coordinator HA, resource‑group punishment, query limits, Alluxio caching, dynamic filtering, and numerous SQL‑level enhancements, with future auto‑scaling and materialized‑view automation.
This article explains why SQL performance degrades as data grows, outlines a step‑by‑step optimization workflow—including slow‑query detection, EXPLAIN analysis, profiling, optimizer tracing, and corrective actions—while detailing type and extra fields, index usage, and practical scenarios such as large pagination, IN‑order‑by, range queries, and complex joins.
This article recounts a series of real‑world challenges—including Excel‑to‑SQL imports, temporary table handling, Canal service failures, jar conflicts, and deployment mistakes—and explains the step‑by‑step solutions that restored system stability and delivered the required data to operations.
This comprehensive guide covers MySQL fundamentals including the three normal forms, permission tables, ACID properties, index design principles, SQL statement categories, sharding, deadlock resolution, isolation anomalies, view characteristics, SQL lifecycle, primary key choices, performance troubleshooting, replication, GTID, backup tools, and backup planning.
This article introduces Apache Kyuubi—a multi‑tenant Thrift JDBC/ODBC service built on Spark—detailing its architecture, advantages over Spark Thrift Server, real‑world use cases, open‑source community progress, and practical deployment strategies on mobile cloud, Kubernetes, and with Trino.
This article explains how to use the nested set model to efficiently query, count, and manage hierarchical department data in MySQL, covering recursive retrieval, leaf detection, insertion, deletion, and traversal techniques with practical SQL examples and JavaScript code for building tree structures.
This article presents a JD.com case study on applying Flink SQL for real‑time dimension modeling, detailing two complex streaming scenarios—full‑join of multiple streams and full‑group aggregation—along with the associated challenges of historical data handling, state management, and performance optimization, and proposes component‑based architectural solutions.
This article explains how to perform MySQL pagination manually in a Java backend using MyBatis by calculating offset and limit, adding total‑count queries, and exposing a REST endpoint that returns paged data and total records without relying on the PageHelper plugin.
This article introduces DBeaver, a free open‑source database management tool, walks through downloading, installing, connecting to various databases, managing tables, using the SQL editor, customizing the UI, generating ER diagrams, and concludes with a summary of its strengths for developers and DBAs.
This article explains how to retrieve sales statistics for multiple business units and their selected products in MySQL, compares loop‑based, OR‑concatenated, mixed‑filter, and SQL‑92 row‑comparison approaches, and shows why the row‑comparison technique was ultimately chosen for its index usage and simplicity.
Sequel Pro is a user‑friendly MySQL client that lets you connect via standard, socket or SSH methods, configure connection details, test and save them, and then explore databases through panels for selecting schemas, viewing table structures, contents, relationships, triggers, table info, and running queries.
This article outlines a systematic approach to identifying and fixing slow MySQL queries, explains how to interpret EXPLAIN output and profiling information, and presents multiple real‑world cases illustrating index usage, implicit conversion, large pagination, IN queries, and other optimization techniques.
This article explains how ClickHouse's AggregatingMergeTree engine uses the special AggregateFunction data type to pre‑aggregate data, demonstrates table creation, data insertion, and querying with state and merge functions, and shows how to combine it with materialized views for efficient analytics.
This article analyzes 2 million Taobao app user‑behavior records using AARRR funnel metrics and RFM segmentation, revealing daily and hourly usage patterns, conversion bottlenecks, product‑search mismatches, and offering data‑driven marketing recommendations to boost retention and sales.
This article explains the purpose and mechanics of PostgreSQL's VACUUM and AUTOVACUUM processes, demonstrates practical examples of table bloat, transaction ID handling, and provides step‑by‑step guidance for precise triggering and performance optimization of vacuum operations.
This article explains why a naïve Python‑based row‑deletion approach is slow for large MySQL tables and provides step‑by‑step SQL techniques—including identifying duplicate names, handling MySQL’s update‑from‑same‑table limitation, and deleting duplicates while preserving a single record per group—complete with executable code examples.
This article presents JD's real-time dimension modeling case using Flink SQL, detailing two complex streaming scenarios, the difficulties of handling historical data and state management, and a component‑based solution that leverages external KV stores and optimized Flink operators to improve performance and scalability.
This article explains how Sohu Smart Media built a high‑performance tracing system for microservices by integrating Zipkin for data collection with StarRocks for storage and analytics, covering architecture, data models, SQL queries, Flink processing, and real‑world results that boost observability and engineering efficiency.
This article compiles essential MySQL interview topics, covering normal forms, privilege tables, ACID properties, SQL statement categories, sharding, deadlocks, isolation anomalies, view characteristics, statement lifecycle, primary key choices, CPU troubleshooting, replication, GTID, backup tools, and backup planning strategies.
This article presents a JD.com BI engineer's case study on applying Flink SQL to real‑time dimension modeling, detailing two complex streaming scenarios, the technical difficulties of handling historical data and performance, and a component‑based solution architecture with future roadmap considerations.
The article explains how ORM bridges Python objects and relational databases, compares options, and justifies choosing the lightweight Peewee library, then details its workflow—model generation, inserting, updating, deleting, and querying records—showcasing how this approach reduces manual data‑creation time from hours to seconds.
This article provides a comprehensive guide to ClickHouse SQL, covering database creation, table definitions, column defaults, temporary and partitioned tables, view types, DDL commands, data import/export formats, and mutation‑based update and delete operations with practical code examples.
This article presents a systematic method for parsing massive MySQL binlog files, identifying which fields are truly updated in each SQL statement, quantifying effective versus ineffective updates, and applying the results to optimize database design and reduce unnecessary binlog generation.
This article explains how to design a MySQL table for product sales statistics and compares several query strategies—including loop queries, OR concatenation, mixed filtering, and the SQL‑92 row‑by‑row comparison—to efficiently retrieve sales data for dynamic sets of business institutions and their products in a single indexed query.
This article examines fifteen common Java pitfalls and best‑practice guidelines—ranging from property copying and date formatting to HashMap sizing, thread‑pool creation, collection handling, logging, and serialization—explaining why each recommendation exists and how it impacts performance and safety.
This article analyzes a production incident where massive OFFSET values in MySQL pagination caused severe query slowdown, demonstrates the problem with simulated data, benchmarks the impact, and presents three practical solutions—including index‑covering subqueries, remembering the last primary key, and limiting offset size—to dramatically improve performance and protect the database.
