Tagged articles

Query Optimization

433 articles · Page 2 of 5
Programmer1970
Programmer1970
Nov 29, 2024 · Databases

40 Essential MySQL Interview Questions and Answers

This article compiles 40 core MySQL interview questions covering ACID properties, lock types, views, stored procedures, JOINs, storage engines, query optimization, buffer pools, write buffers, transaction mechanisms, MVCC, isolation levels, gap locks, metadata locks, thread model, row formats, and more, providing concise answers for each.

DatabaseMVCCMySQL
0 likes · 57 min read
40 Essential MySQL Interview Questions and Answers
Ctrip Technology
Ctrip Technology
Nov 21, 2024 · Big Data

Performance Governance and Optimization of Ctrip's Nova Data Reporting Platform

This article details the performance challenges of Ctrip's Nova data reporting platform and describes a series of governance measures—including multi‑dimensional data caching, materialized view acceleration, query strategy optimization, and SQL quality improvements—that collectively reduced average query latency by over 50% and stabilized the system.

Query OptimizationSQL Performancecaching
0 likes · 26 min read
Performance Governance and Optimization of Ctrip's Nova Data Reporting Platform
Aikesheng Open Source Community
Aikesheng Open Source Community
Nov 11, 2024 · Databases

New JSON Format for EXPLAIN and EXPLAIN ANALYZE in MySQL 8.3+

MySQL 8.3 introduces a new iterator‑aware JSON output for EXPLAIN and EXPLAIN ANALYZE, selectable via the explain_json_format_version variable, which aligns JSON with the tree format, provides richer execution statistics, and can be accessed programmatically through EXPLAIN INTO across all recent MySQL releases.

EXPLAINMySQLQuery Optimization
0 likes · 11 min read
New JSON Format for EXPLAIN and EXPLAIN ANALYZE in MySQL 8.3+
Shopee Tech Team
Shopee Tech Team
Oct 25, 2024 · Big Data

StarRocks at Shopee: Practical Use Cases and Performance Analysis

Shopee’s deployment of StarRocks across DataService, DataGo, and DataStudio demonstrates that its vectorized engine, cost‑based optimizer, and materialized‑view caching can query Hive, Iceberg, Delta Lake and Hudi up to 20,000× faster than Presto, cutting CPU usage and delivering consistently lower latency for complex analytics.

Data LakeHiveMPP
0 likes · 11 min read
StarRocks at Shopee: Practical Use Cases and Performance Analysis
Java Architect Essentials
Java Architect Essentials
Oct 23, 2024 · Databases

Does Varchar Length Really Affect MySQL Storage and Performance?

This article investigates whether MySQL varchar column length influences storage size and query performance by creating two tables with varchar(50) and varchar(500), inserting one million rows, measuring disk usage, and benchmarking various queries, ultimately revealing that storage is identical while longer columns can degrade performance during sorted scans due to increased temporary file usage.

IndexMySQLQuery Optimization
0 likes · 10 min read
Does Varchar Length Really Affect MySQL Storage and Performance?
Baidu Tech Salon
Baidu Tech Salon
Oct 22, 2024 · Big Data

TDE-ClickHouse: Baidu MEG's High-Performance Big Data Analytics Engine

TDE‑ClickHouse, the core engine of Baidu MEG’s Turing 3.0 ecosystem, delivers sub‑second, self‑service analytics on petabyte‑scale data by decoupling compute, adding multi‑level aggregation, high‑cardinality and rule‑based optimizations, a two‑phase bulk‑load pipeline, cloud‑native deployment, and a lightweight meta service, now powering over 350 000 cores, 10 PB storage and more than 150 000 daily BI queries with average response times under three seconds.

ClickHouseDatabase ArchitectureQuery Optimization
0 likes · 19 min read
TDE-ClickHouse: Baidu MEG's High-Performance Big Data Analytics Engine
Code Ape Tech Column
Code Ape Tech Column
Oct 21, 2024 · Big Data

Design and Optimization of Querying 100k Records from Tens of Millions Using ClickHouse, Elasticsearch, HBase, and RediSearch

This article presents a business-driven requirement to extract no more than 100,000 records from a pool of tens of millions, evaluates four technical solutions—including multithreaded ClickHouse pagination, Elasticsearch scroll‑scan, an ES‑HBase hybrid, and RediSearch + RedisJSON—provides implementation details, performance measurements, and practical recommendations for large‑scale data querying.

Big DataHBaseQuery Optimization
0 likes · 11 min read
Design and Optimization of Querying 100k Records from Tens of Millions Using ClickHouse, Elasticsearch, HBase, and RediSearch
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Oct 19, 2024 · Databases

Understanding the HAVING Clause in SQL: Concepts, Examples, and Best Practices

This article explains the purpose and proper use of the SQL HAVING clause, contrasts it with WHERE, and provides multiple practical examples—including counting groups, detecting missing IDs, calculating mode and median, and filtering fully‑submitted records—while highlighting common pitfalls and performance considerations.

AggregationDatabaseGROUP BY
0 likes · 13 min read
Understanding the HAVING Clause in SQL: Concepts, Examples, and Best Practices
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 16, 2024 · Databases

Kuaishou's Lakehouse‑Integrated OLAP Architecture with Apache Doris: Design, Migration, and Optimization

The article describes how Kuaishou transformed its high‑traffic OLAP system from a separated lake‑and‑warehouse architecture using Hive/Hudi and ClickHouse into a unified lakehouse solution powered by Apache Doris, detailing the challenges, design choices, caching and automatic materialization mechanisms, and the resulting performance and governance improvements.

Apache DorisBig DataData Caching
0 likes · 18 min read
Kuaishou's Lakehouse‑Integrated OLAP Architecture with Apache Doris: Design, Migration, and Optimization
Programmer1970
Programmer1970
Sep 26, 2024 · Databases

How MySQL’s Multi-Range Read (MRR) Turns Random I/O into Sequential I/O

The article explains MySQL’s Multi-Range Read (MRR) optimization, detailing how it scans secondary indexes, buffers and sorts primary keys, leverages disk pre‑read and cache locality to convert random I/O into sequential I/O, and shows configuration, monitoring methods, and a concrete orders‑table example.

Disk I/OInnoDBMRR
0 likes · 11 min read
How MySQL’s Multi-Range Read (MRR) Turns Random I/O into Sequential I/O
dbaplus Community
dbaplus Community
Sep 25, 2024 · Databases

Segment‑wise Train Seat Allocation with SQL: A Practical Guide

The article recounts a personal experience of a failed train ticket change, analyzes segment‑wise seat availability, proposes a database model, and demonstrates a SQL query that allocates seats per segment, showing how such logic could enable full‑journey booking and even inspired a new feature on the 12306 platform.

DatabaseOracleQuery Optimization
0 likes · 10 min read
Segment‑wise Train Seat Allocation with SQL: A Practical Guide
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 18, 2024 · Databases

Doris Performance Optimization: OLAP Query, Indexes, Vectorized Execution, and High‑Concurrency Point Queries

This article explains how Apache Doris achieves high‑concurrency OLAP and point‑query performance through MPP architecture, columnar storage, partition‑bucket pruning, various indexes, materialized views, vectorized execution, runtime filters, short‑circuit planning, and prepared‑statement caching.

DorisOLAPQuery Optimization
0 likes · 12 min read
Doris Performance Optimization: OLAP Query, Indexes, Vectorized Execution, and High‑Concurrency Point Queries
dbaplus Community
dbaplus Community
Sep 10, 2024 · Databases

Boost MySQL Performance: Essential Tools, Installation & Report Analysis

This guide introduces four popular MySQL performance‑tuning utilities—mysqltuner.pl, tuning‑primer.sh, pt‑variable‑advisor, and pt‑query‑digest—provides download commands, usage examples, and detailed tips for interpreting their diagnostic reports to improve database configuration and query efficiency.

Database ToolsMySQLPerformance Tuning
0 likes · 8 min read
Boost MySQL Performance: Essential Tools, Installation & Report Analysis
ITPUB
ITPUB
Sep 3, 2024 · Backend Development

Boost Query Speed from 20 s to 0.5 s with MQ‑Driven Elasticsearch Separation (Java)

A large table with tens of millions of rows caused 20‑second query times, but by decoupling reads through RabbitMQ and writing data asynchronously to Elasticsearch, query latency dropped to 500 ms, with Java code examples for both producer and consumer, sync handling, and migration tips.

ElasticsearchJavaQuery Optimization
0 likes · 6 min read
Boost Query Speed from 20 s to 0.5 s with MQ‑Driven Elasticsearch Separation (Java)
Java Architect Essentials
Java Architect Essentials
Aug 9, 2024 · Databases

Comprehensive MySQL Query and Configuration Optimization Guide

This article provides a thorough guide to MySQL performance optimization, covering general query improvements, proper data types, index usage, configuration parameter tuning, avoiding common pitfalls, and detailed EXPLAIN analysis, with concrete SQL examples and practical recommendations for developers and DBAs.

IndexingMySQLPerformance Tuning
0 likes · 16 min read
Comprehensive MySQL Query and Configuration Optimization Guide
Bilibili Tech
Bilibili Tech
Aug 9, 2024 · Operations

Design and Optimization of Monitoring 2.0 Architecture with VictoriaMetrics and Flink

The new Monitoring 2.0 architecture separates collection, compute and storage, adopts VictoriaMetrics for compact time‑series storage and a zone‑based scheduler, introduces push‑based ingestion, uses Flink for real‑time pre‑aggregation and automatic PromQL rewrite, delivering ten‑fold query speedups, sub‑300 ms p90 latency, and dramatically higher write and query throughput.

FlinkMetricsMonitoring
0 likes · 29 min read
Design and Optimization of Monitoring 2.0 Architecture with VictoriaMetrics and Flink
Java Captain
Java Captain
Jul 29, 2024 · Information Security

How to Perform Fuzzy Queries on Encrypted Data

This article examines the challenges of fuzzy searching encrypted data and compares three categories of solutions—naïve, conventional, and advanced—detailing their implementation ideas, performance trade‑offs, storage costs, and security implications for real‑world applications.

EncryptionFuzzy SearchQuery Optimization
0 likes · 10 min read
How to Perform Fuzzy Queries on Encrypted Data
Java Interview Crash Guide
Java Interview Crash Guide
Jul 24, 2024 · Databases

How to Supercharge MySQL Queries: 8 Proven Optimization Techniques

Learn eight practical MySQL optimization strategies—including smarter LIMIT usage, avoiding implicit type conversion, rewriting subqueries with JOINs, handling mixed sorting, eliminating costly EXISTS clauses, pushing conditions, pre‑filtering data, and leveraging intermediate result sets—to dramatically reduce query execution time from seconds to milliseconds.

IndexingLIMITMySQL
0 likes · 15 min read
How to Supercharge MySQL Queries: 8 Proven Optimization Techniques
Selected Java Interview Questions
Selected Java Interview Questions
Jul 16, 2024 · Databases

SQL Query Optimization Techniques

This article explains how to speed up SQL queries while keeping statements concise, covering the query processing workflow, optimization goals, and practical tips such as selecting specific columns, avoiding DISTINCT, using proper indexes, preferring EXISTS over COUNT, limiting result sets, favoring WHERE over HAVING, and replacing correlated subqueries with joins.

DatabaseQuery OptimizationSQL
0 likes · 9 min read
SQL Query Optimization Techniques
Sohu Tech Products
Sohu Tech Products
Jul 10, 2024 · Industry Insights

How StarRocks and Apache Paimon Transform Data Lake Analytics and Migration

This article provides a practical deep‑dive into StarRocks and Apache Paimon, covering data‑lake fundamentals, the technical advantages of both platforms, performance gains over traditional engines, step‑by‑step migration strategies, deployment options on Alibaba Cloud EMR, and future roadmap plans.

Apache PaimonData LakeQuery Optimization
0 likes · 15 min read
How StarRocks and Apache Paimon Transform Data Lake Analytics and Migration
Java Tech Enthusiast
Java Tech Enthusiast
Jul 7, 2024 · Databases

MySQL Query Optimization and Performance Tuning Guide

This guide presents a thorough MySQL performance checklist, covering efficient SQL writing, data‑type choices, selective column retrieval, smart joins, batch inserts, and query‑cache settings; it details key InnoDB and server variable tuning, proper index creation and usage, common pitfalls, and how to analyze and improve queries with EXPLAIN.

DatabaseMySQLQuery Optimization
0 likes · 17 min read
MySQL Query Optimization and Performance Tuning Guide
Tencent Cloud Developer
Tencent Cloud Developer
Jul 5, 2024 · Databases

Database Systems Reading List: Fundamentals, Design, Engines, and Optimizations

This curated bibliography compiles seminal database‑system papers—spanning foundational relational theory, system design for RDBMS and NoSQL, SQL engine optimization, storage structures, transaction and scheduling mechanisms, plus workload, networking, quality, and tuning research—each entry listing title, authors, year and a PDF link for scholars and practitioners.

Database SystemsQuery OptimizationStorage Engine
0 likes · 25 min read
Database Systems Reading List: Fundamentals, Design, Engines, and Optimizations
Python Programming Learning Circle
Python Programming Learning Circle
Jun 11, 2024 · Databases

SQL Best Practices: Avoid Common Mistakes and Write Efficient Queries

This article presents a reverse‑analysis of common SQL pitfalls—such as ambiguous column names, selecting all columns, unnecessary joins, missing WHERE clauses, lack of ORDER BY, unclear aliases, absent error handling, redundant subqueries, and missing comments—and provides clear "Don’t" and "Do" examples with code to help you write clean, optimized, and reliable SQL statements.

Query Optimizationbest practiceserror-handling
0 likes · 7 min read
SQL Best Practices: Avoid Common Mistakes and Write Efficient Queries
Huawei Cloud Developer Alliance
Huawei Cloud Developer Alliance
Jun 3, 2024 · Databases

Master OpenGemini: From Schema Design to Performance Tuning Best Practices

This article summarizes a live session where Shawn explains how understanding business scenarios drives effective OpenGemini database design, and provides comprehensive best‑practice guidance on library and table design, data ingestion, query optimization, and performance tuning for time‑series workloads.

Data IngestionPerformance TuningQuery Optimization
0 likes · 7 min read
Master OpenGemini: From Schema Design to Performance Tuning Best Practices
Top Architect
Top Architect
May 24, 2024 · Databases

How to Perform Fuzzy Search on Encrypted Data

This article examines various techniques for enabling fuzzy queries on encrypted fields, comparing naive memory‑based methods, conventional token‑based approaches that leverage database indexes, and advanced cryptographic schemes, and recommends practical solutions for real‑world applications.

Fuzzy SearchQuery Optimizationdatabase encryption
0 likes · 12 min read
How to Perform Fuzzy Search on Encrypted Data
Lobster Programming
Lobster Programming
Apr 28, 2024 · Databases

How to Solve MySQL Deep Pagination: Causes and 6 Effective Optimization Strategies

Deep pagination in MySQL becomes a performance bottleneck when tables reach millions of rows, causing excessive row scans and costly index lookups; this article explains its root causes and presents six practical optimization techniques—including keyset pagination, delayed joins, covering indexes, data partitioning, and external search engines—to dramatically improve query speed.

Covering IndexDeep PaginationIndexing
0 likes · 7 min read
How to Solve MySQL Deep Pagination: Causes and 6 Effective Optimization Strategies
StarRocks
StarRocks
Apr 18, 2024 · Databases

Master StarRocks Deployment and Optimization: From Capacity Planning to Query Tuning

This comprehensive guide walks you through StarRocks production deployment, covering capacity planning, hardware specs, environment setup, table modeling, partitioning, bucketing, index selection, data import best practices, query optimization, monitoring, and resource‑group configuration, all backed by concrete SQL examples and configuration commands.

ModelingQuery OptimizationStarRocks
0 likes · 20 min read
Master StarRocks Deployment and Optimization: From Capacity Planning to Query Tuning
ITPUB
ITPUB
Apr 11, 2024 · Big Data

Query 100K Items from 10M+ Records: CK, ES Scroll, HBase, RediSearch

When faced with a business requirement to filter up to 100 000 records from a pool of tens of millions and then sort and de‑duplicate them, this article explores four technical solutions—multithreaded ClickHouse pagination, Elasticsearch scroll‑scan, a combined Elasticsearch‑HBase approach, and RediSearch with RedisJSON—detailing their design, implementation, performance testing, and trade‑offs.

Big DataClickHouseElasticsearch
0 likes · 12 min read
Query 100K Items from 10M+ Records: CK, ES Scroll, HBase, RediSearch
dbaplus Community
dbaplus Community
Apr 10, 2024 · Databases

How MySQL Chooses the Cheapest Index for COUNT(*) and When It Gets It Wrong

This article examines whether SELECT COUNT(*) causes full‑table scans, explains MySQL’s optimizer cost‑based index selection (including IO and CPU costs), demonstrates with a 100k‑row table how auxiliary indexes are chosen, and shows cases where the optimizer’s estimates mislead performance.

COUNTIndex CostMySQL
0 likes · 9 min read
How MySQL Chooses the Cheapest Index for COUNT(*) and When It Gets It Wrong
Baidu Geek Talk
Baidu Geek Talk
Apr 10, 2024 · Big Data

TDA: A One‑Stop Self‑Service BI Platform – Architecture, Challenges, and Solutions

The article presents Turing Data Analysis (TDA), a self‑service BI platform that replaces fragile traditional pipelines with a unified DWD‑based data model, drag‑and‑drop analytics, multi‑engine query optimization and caching, delivering sub‑10‑second queries on billions of rows, fine‑grained permissions, and rapid dashboard creation, while reporting significant usage growth and outlining AI‑driven future enhancements.

BIBig DataMPP
0 likes · 15 min read
TDA: A One‑Stop Self‑Service BI Platform – Architecture, Challenges, and Solutions
DataFunTalk
DataFunTalk
Mar 16, 2024 · Big Data

Performance Optimization Practices for KwaiBI Big Data Analysis Platform

This article introduces KwaiBI, the internal data analysis product of Kuaishou, outlines its five major functional areas, details the performance challenges of large‑scale analytics, and presents a comprehensive set of optimization techniques—including cache warming, query rewriting, materialized acceleration, and the Bleem lake‑house engine—along with future directions and a brief Q&A.

Big DataData AnalyticsKwaiBI
0 likes · 15 min read
Performance Optimization Practices for KwaiBI Big Data Analysis Platform
Java Tech Enthusiast
Java Tech Enthusiast
Mar 10, 2024 · Databases

Why IN and NOT IN Should Be Avoided in SQL Queries

Because IN and NOT IN often bypass indexes on large tables, can yield incorrect results when column names differ or NULL values are present, and may cause severe performance penalties, they should be replaced with EXISTS/NOT EXISTS subqueries or appropriate JOINs, reserving IN only for small literal lists.

IN ClauseQuery OptimizationSQL
0 likes · 4 min read
Why IN and NOT IN Should Be Avoided in SQL Queries
MaGe Linux Operations
MaGe Linux Operations
Feb 12, 2024 · Databases

Why Hash Join Beats Nested Loop Join and When It Fails

This article explains why hash joins usually outperform nested‑loop joins, how to force hash joins in SQL, the data‑type restrictions that prevent hash joins, and practical tips for optimizing join performance in TD and Oracle compatibility modes.

Data TypesHash JoinNested Loop
0 likes · 6 min read
Why Hash Join Beats Nested Loop Join and When It Fails
Sohu Tech Products
Sohu Tech Products
Jan 24, 2024 · Databases

Optimizing Database Expression Evaluation with JIT Technology Using Gandiva

The article explains how database expression evaluation—especially in WHERE and SELECT clauses—can be dramatically accelerated by replacing interpreted AST traversal with Just‑In‑Time compilation using Apache Gandiva, which leverages LLVM to generate SIMD‑optimized machine code for Arrow columnar data, and discusses extensions such as timestamp, array, higher‑order functions, and UDF support.

Apache ArrowApache GandivaExpression Evaluation
0 likes · 17 min read
Optimizing Database Expression Evaluation with JIT Technology Using Gandiva
ITPUB
ITPUB
Jan 8, 2024 · Databases

How I Cut an 8‑Second SQL Query to 0.7 Seconds: Step‑by‑Step Optimization

This article walks through a real‑world case of a slow MySQL query that took 8 seconds, explains how the original use of EXISTS and NOT EXISTS caused performance bottlenecks, and details two successive rewrites—first using JOIN + GROUP BY, then replacing NOT EXISTS with NOT IN—bringing execution time down to 0.7 seconds while highlighting key indexing and testing lessons.

DatabaseMySQLPerformance Tuning
0 likes · 7 min read
How I Cut an 8‑Second SQL Query to 0.7 Seconds: Step‑by‑Step Optimization
DeWu Technology
DeWu Technology
Dec 18, 2023 · Databases

Elasticsearch Best Practices: Query, Index, and Performance Optimizations

The guide outlines production‑ready Elasticsearch best practices, covering query tuning such as using shard request cache, filter context, size‑0 aggregations and composite aggregations; write strategies like auto‑generated IDs, bulk API sizing and refresh handling; optimal shard counts, explicit mappings with disabled unnecessary features, and general advice to use explicit index names and stored scripts.

ElasticsearchIndexingQuery Optimization
0 likes · 22 min read
Elasticsearch Best Practices: Query, Index, and Performance Optimizations
Architecture Digest
Architecture Digest
Dec 6, 2023 · Databases

MySQL Query Optimization and Index Tuning: From Subqueries to Joins

This article analyzes a slow MySQL query that scans millions of rows to find students scoring 100 in Chinese, then demonstrates how adding appropriate single‑column and composite indexes, rewriting the query as a join, and using covering indexes can reduce execution time from hours to milliseconds.

MySQLPerformance TuningQuery Optimization
0 likes · 10 min read
MySQL Query Optimization and Index Tuning: From Subqueries to Joins
dbaplus Community
dbaplus Community
Dec 5, 2023 · Databases

How I Cut an 8‑Second Slow Query to 0.7 Seconds with Two Optimizations

The article walks through a real‑world case of a slow MySQL query that took 8 seconds, explains why simple indexing wasn't enough, and demonstrates two successive rewrites using JOIN‑GROUP BY and NOT IN that reduced execution time to 0.7 seconds while detailing the EXPLAIN analysis.

EXPLAINMySQLPerformance Tuning
0 likes · 6 min read
How I Cut an 8‑Second Slow Query to 0.7 Seconds with Two Optimizations
JD Cloud Developers
JD Cloud Developers
Nov 14, 2023 · Databases

How MySQL Executes a Simple SQL Query: Step‑by‑Step Deep Dive

This article walks through the complete lifecycle of a MySQL query—from the client request, through connection handling, parsing, optimization, execution, and storage engine processing—illustrating each component with diagrams, code snippets, and a detailed analysis of execution order for both simple and join queries.

Database InternalsMySQLQuery Optimization
0 likes · 12 min read
How MySQL Executes a Simple SQL Query: Step‑by‑Step Deep Dive
ITPUB
ITPUB
Nov 1, 2023 · Databases

Doris 2.0.2 vs 1.2.3: Real‑World Query Performance Comparison

After upgrading a Doris cluster from version 1.2.3 to 2.0.2, the author runs a series of SQL benchmarks—including PK lookups, top‑client queries, distinct counts on low‑ and high‑cardinality columns, minute‑level session analysis, and full‑table deduplication—to measure execution times, revealing mixed performance gains and regressions across the seven test scenarios.

Database UpgradeDorisQuery Optimization
0 likes · 9 min read
Doris 2.0.2 vs 1.2.3: Real‑World Query Performance Comparison
Java Architect Essentials
Java Architect Essentials
Oct 17, 2023 · Databases

Understanding LEFT JOIN ON vs WHERE Conditions in SQL

This article explains how LEFT JOIN works in SQL, clarifying the difference between ON‑clause conditions that affect the join process and WHERE‑clause filters that operate on the resulting temporary table, using multiple example queries and visual illustrations.

DatabaseLeft JoinON clause
0 likes · 5 min read
Understanding LEFT JOIN ON vs WHERE Conditions in SQL
ITPUB
ITPUB
Sep 28, 2023 · Databases

Mastering MySQL Index Merge: How Three Algorithms Boost Query Performance

Learn how MySQL's index merge optimization works, explore its three algorithms—intersect, union, and sort_union—understand when each applies, see practical setup and sample queries, and discover configuration flags and best‑practice tips to improve query efficiency.

DatabaseMySQLQuery Optimization
0 likes · 9 min read
Mastering MySQL Index Merge: How Three Algorithms Boost Query Performance
DataFunSummit
DataFunSummit
Sep 25, 2023 · Big Data

Trino in Bilibili Lakehouse: Compute Engine, Stability, and Containerization Practices

This article presents Bilibili's practical implementation of Trino within a lakehouse architecture, focusing on the compute engine placement, stability enhancements, and containerized deployment, while detailing indexing strategies, pre‑computation techniques, Iceberg metadata optimizations, and performance gains for large‑scale analytical queries.

IcebergIndexingLakehouse
0 likes · 14 min read
Trino in Bilibili Lakehouse: Compute Engine, Stability, and Containerization Practices
JD Retail Technology
JD Retail Technology
Sep 13, 2023 · Databases

Optimizing Pagination Queries for Billion‑Row MySQL Tables

The article analyzes the performance problems of deep pagination on massive MySQL tables storing billions of fan records and presents three progressive solutions—simple LIMIT, tag‑record (maxId) pagination, and range‑limited pagination with async and offline minId caching—along with general indexing best‑practices for high‑throughput queries.

IndexingLarge DataMySQL
0 likes · 10 min read
Optimizing Pagination Queries for Billion‑Row MySQL Tables
Liangxu Linux
Liangxu Linux
Sep 10, 2023 · Databases

Master MySQL Query Optimization: Architecture, Caching, and Index Strategies

This article explains MySQL's logical architecture, query execution flow, caching mechanisms, and detailed index design techniques, providing practical tips and examples so developers can understand the underlying principles and apply effective performance optimizations in real‑world applications.

Database PerformanceMySQLQuery Optimization
0 likes · 38 min read
Master MySQL Query Optimization: Architecture, Caching, and Index Strategies
Ximalaya Technology Team
Ximalaya Technology Team
Sep 6, 2023 · Backend Development

Design Analysis of Lucene and In-Memory Inverted Index Service for Advertising Retrieval

The team analyzed Lucene’s disk‑based inverted index and built a custom in‑memory inverted‑index service for Himalaya’s ad engine, encoding terms as 64‑bit keys, supporting real‑time updates and BooleanQuery‑style and custom expression filtering, which cut query latency from ~50 ms to under 5 ms and enabled massive scaling.

Data StructuresJavaLucene
0 likes · 27 min read
Design Analysis of Lucene and In-Memory Inverted Index Service for Advertising Retrieval
Liangxu Linux
Liangxu Linux
Sep 4, 2023 · Databases

What Really Happens Inside SQL? A Step‑by‑Step Walkthrough of Execution Order

This article breaks down the true execution sequence of an SQL query—from FROM and JOIN through WHERE, GROUP BY, HAVING, SELECT, DISTINCT, ORDER BY, and finally LIMIT—explaining each stage with examples, code snippets, and visual diagrams to help developers write more efficient queries.

DatabaseExecution OrderQuery Optimization
0 likes · 7 min read
What Really Happens Inside SQL? A Step‑by‑Step Walkthrough of Execution Order
MaGe Linux Operations
MaGe Linux Operations
Aug 30, 2023 · Databases

Understanding SQL Execution Order: From FROM to LIMIT Explained

This article walks through the exact sequence a database follows when executing a SQL query—starting with FROM and JOIN, then WHERE, GROUP BY, HAVING, SELECT, DISTINCT, ORDER BY, and finally LIMIT—highlighting key differences from the textual order of the statement.

DatabaseExecution OrderQuery Optimization
0 likes · 5 min read
Understanding SQL Execution Order: From FROM to LIMIT Explained
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Aug 16, 2023 · Databases

Common Cases When MySQL Indexes Are Ignored

This article explains how to set up a MySQL test table and indexes, then enumerates nine typical scenarios—such as leading‑wildcard LIKE, arithmetic or functions on indexed columns, type mismatches, composite‑index misuse, character‑set differences, and OR conditions with non‑indexed fields—that cause queries to bypass indexes and perform full table scans.

DatabaseIndexMySQL
0 likes · 8 min read
Common Cases When MySQL Indexes Are Ignored
Java Architect Essentials
Java Architect Essentials
Aug 13, 2023 · Information Security

How to Perform Fuzzy Searches on Encrypted Data Without Breaking Security

This article examines three categories of approaches—naïve, conventional, and advanced—for enabling fuzzy queries on encrypted fields, comparing their implementation steps, performance trade‑offs, storage costs, and security implications, and provides practical examples such as in‑memory decryption, tag mapping, database functions, tokenization, and algorithm‑level designs.

Database SecurityEncryptionFuzzy Search
0 likes · 11 min read
How to Perform Fuzzy Searches on Encrypted Data Without Breaking Security
dbaplus Community
dbaplus Community
Aug 13, 2023 · Databases

Why Does This PostgreSQL Self‑Join Behave Unexpectedly? InitPlan & One‑Time Filter

The article examines a puzzling PostgreSQL self‑join query that yields surprising true/false results, walks through step‑by‑step rewrites, analyzes the resulting execution plans—including the appearance of InitPlan and One‑Time Filter nodes—and explains how correlated versus uncorrelated subqueries affect query evaluation.

InitPlanPostgreSQLQuery Optimization
0 likes · 11 min read
Why Does This PostgreSQL Self‑Join Behave Unexpectedly? InitPlan & One‑Time Filter
Top Architect
Top Architect
Aug 8, 2023 · Databases

MySQL Pagination Query Optimization Techniques and Performance Testing

The article examines various MySQL pagination strategies—including simple LIMIT clauses, sub‑query approaches, ID‑range filtering, and temporary‑table methods—provides performance measurements on a table with millions of rows, and offers practical recommendations for reducing query latency in large datasets.

DatabaseMySQLPagination
0 likes · 9 min read
MySQL Pagination Query Optimization Techniques and Performance Testing
DataFunTalk
DataFunTalk
Jul 30, 2023 · Databases

MatrixOne Architecture and OLAP Engine Design Overview

This article presents an in‑depth overview of MatrixOne, an open‑source hyper‑converged cloud‑native database, detailing its three‑tier architecture of compute, data and file services, and explains the design and implementation of its OLAP engine, including parser, planner, optimizer, and push‑based execution pipeline.

Database ArchitectureExecution EngineMatrixOne
0 likes · 18 min read
MatrixOne Architecture and OLAP Engine Design Overview
21CTO
21CTO
Jul 17, 2023 · Big Data

How WeChat Cut Query Latency from Seconds to 100 ms with Druid Optimizations

This case study explains how the WeChat multi‑dimensional monitoring platform identified performance bottlenecks in its Druid‑based data layer, analyzed user query patterns, and applied sub‑query splitting, Redis caching, and segment size reductions to achieve over 85% cache‑hit rates and bring average query latency down to around 100 ms.

Big DataDruidMonitoring
0 likes · 13 min read
How WeChat Cut Query Latency from Seconds to 100 ms with Druid Optimizations
Su San Talks Tech
Su San Talks Tech
Jul 8, 2023 · Databases

Master MySQL Index Optimization: 20 Proven Rules to Boost Query Performance

This comprehensive guide explains 20 practical MySQL index optimization rules—including when NOT to use indexes, how to choose the right index type, and techniques such as prefix indexes, covering indexes, and query rewriting—to dramatically improve SQL query efficiency and reduce database load.

MySQLQuery OptimizationSQL Performance
0 likes · 19 min read
Master MySQL Index Optimization: 20 Proven Rules to Boost Query Performance
Laravel Tech Community
Laravel Tech Community
Jul 5, 2023 · Databases

Performance Testing and Optimization of Large‑Scale MySQL Queries

This article demonstrates how to generate, insert, and query millions of rows in a MySQL 5.7 table, measures pagination performance under varying offsets and result sizes, and presents several optimization techniques—including sub‑queries, indexed look‑ups, and column selection—to dramatically reduce query latency.

DatabaseLarge DataMySQL
0 likes · 10 min read
Performance Testing and Optimization of Large‑Scale MySQL Queries
Big Data Technology Architecture
Big Data Technology Architecture
Jul 4, 2023 · Databases

Apache Doris 2.0‑beta Release: New Query Optimizer, Pipeline Engine, Workload Management and Performance Enhancements

Apache Doris 2.0‑beta, released on July 3, 2023, introduces a modern Cascades‑based query optimizer, a data‑driven pipeline execution engine, fine‑grained workload groups, enhanced memory management, partial‑column updates, compute nodes, cold‑hot tiering and cross‑cluster replication, delivering up to tenfold speedups and significant cost reductions for real‑time analytics.

Apache DorisDatabasePipeline Execution
0 likes · 24 min read
Apache Doris 2.0‑beta Release: New Query Optimizer, Pipeline Engine, Workload Management and Performance Enhancements
dbaplus Community
dbaplus Community
Jun 25, 2023 · Big Data

WeChat’s 10× Query Speedup: From 1000ms to 100ms with Druid & Redis

WeChat’s multi‑dimensional monitoring platform faced severe query latency and I/O bottlenecks, so the team analyzed user behavior and Druid architecture, then introduced sub‑query splitting, Redis caching, and segment size reductions, achieving over 85% cache hit rate and reducing average query time to around 100 ms.

Big DataCacheDruid
0 likes · 12 min read
WeChat’s 10× Query Speedup: From 1000ms to 100ms with Druid & Redis
Architect
Architect
Jun 23, 2023 · Big Data

Optimizing Query Performance in WeChat's Multi‑Dimensional Monitoring Platform

This article details how the WeChat multi‑dimensional monitoring platform reduced average query latency from over 1000 ms to around 100 ms by analyzing user query patterns, redesigning the Druid data layer, splitting sub‑queries, introducing Redis caching, and employing sub‑dimension tables, achieving cache hit rates above 85%.

DruidMonitoringQuery Optimization
0 likes · 13 min read
Optimizing Query Performance in WeChat's Multi‑Dimensional Monitoring Platform
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 13, 2023 · Big Data

Iceberg Data Lake Implementation and Optimization at iQIYI

This article details iQIYI's adoption of Iceberg for its data lake, covering the OLAP architecture, reasons for a data lake, Iceberg's table format advantages over Hive, platform construction, streaming ingestion, query and performance optimizations, real‑world business deployments, and future plans.

Big DataData LakeFlink
0 likes · 21 min read
Iceberg Data Lake Implementation and Optimization at iQIYI
FunTester
FunTester
Jun 7, 2023 · Big Data

Optimizing Query Performance in WeChat's Multi‑Dimensional Monitoring Platform with Druid and Redis

The article details how WeChat's multi‑dimensional metric monitoring platform, which handles billions of data points per minute, reduced average query latency from over 1000 ms to around 140 ms and achieved over 85% cache hit rate by analyzing query behavior, redesigning the data layer architecture, splitting queries into sub‑queries, adding Redis caching, and introducing sub‑dimension tables.

Big DataCacheDruid
0 likes · 13 min read
Optimizing Query Performance in WeChat's Multi‑Dimensional Monitoring Platform with Druid and Redis
Java High-Performance Architecture
Java High-Performance Architecture
Jun 5, 2023 · Databases

Master MySQL Indexes: Types, Usage, and Performance Optimization

This guide explains MySQL indexes—what they are, their data structures, advantages and drawbacks, when to create them, various index types, how to interpret EXPLAIN output, common pitfalls that cause index loss, and practical optimization techniques for queries, joins, ordering, grouping, and slow‑query logging.

Database PerformanceEXPLAINIndex
0 likes · 19 min read
Master MySQL Indexes: Types, Usage, and Performance Optimization
Top Architect
Top Architect
Jun 1, 2023 · Databases

Comprehensive Guide to MySQL Indexes, Types, and Query Optimization

This article explains the definition, data structures, advantages, disadvantages, and appropriate usage scenarios of MySQL indexes, details primary, unique, single‑column and composite index types, analyzes common performance bottlenecks with EXPLAIN, demonstrates index creation, invalidation cases, query and join optimizations, and shows how to configure and use the slow‑query log.

Database PerformanceEXPLAINMySQL
0 likes · 19 min read
Comprehensive Guide to MySQL Indexes, Types, and Query Optimization
DataFunTalk
DataFunTalk
May 30, 2023 · Big Data

Optimizing Chart Query Performance in YouShu BI: Data Query Principles, Intelligent Caching, Query Merging, and Diagnostics

This article explains the data query fundamentals of YouShu BI charts, introduces intelligent caching design, describes query merging and various optimization techniques—including partition filters, value acceleration, and SQL generation—and outlines performance diagnosis methods to improve BI chart responsiveness.

BIBig DataChart Performance
0 likes · 16 min read
Optimizing Chart Query Performance in YouShu BI: Data Query Principles, Intelligent Caching, Query Merging, and Diagnostics
php Courses
php Courses
May 26, 2023 · Databases

Using Indexes to Optimize MySQL Queries

This article explains what indexes are, when they should be applied, and provides step‑by‑step MySQL commands—including single‑column, composite, and index‑hint examples—to improve query performance on large datasets.

Database PerformanceIndexingMySQL
0 likes · 6 min read
Using Indexes to Optimize MySQL Queries
Programmer DD
Programmer DD
May 24, 2023 · Databases

Why SQL Struggles with Complex Queries and How SPL Offers a Solution

This article examines the inherent difficulties of SQL when handling multi‑step, ordered, and object‑referencing queries, illustrates them with practical examples, and introduces the Structured Process Language (SPL) as a more expressive alternative for data processing tasks.

CTEQuery OptimizationSPL
0 likes · 16 min read
Why SQL Struggles with Complex Queries and How SPL Offers a Solution
Sanyou's Java Diary
Sanyou's Java Diary
May 23, 2023 · Databases

Master MySQL Indexes: From B+ Trees to Index Merge and Optimization

This article provides a comprehensive guide to MySQL indexing, covering index types, data structures, clustering vs. non‑clustering, hash and B+Tree indexes, covering indexes, index push‑down, index merge strategies, cost‑based index selection, common pitfalls, and practical indexing principles with illustrative examples and SQL snippets.

B+TreeDatabaseIndex
0 likes · 31 min read
Master MySQL Indexes: From B+ Trees to Index Merge and Optimization
iQIYI Technical Product Team
iQIYI Technical Product Team
May 12, 2023 · Operations

Performance Troubleshooting and Optimization of Prometheus Monitoring Queries

The article explains that high metric cardinality in Prometheus causes long query times and timeouts, and demonstrates how using recording rules to pre‑compute aggregates dramatically reduces cardinality and latency, while recommending scrape interval tuning and metric design best practices to keep charts responsive.

PrometheusQuery OptimizationRecording Rules
0 likes · 10 min read
Performance Troubleshooting and Optimization of Prometheus Monitoring Queries
dbaplus Community
dbaplus Community
May 7, 2023 · Databases

Mastering MySQL Indexes: 15 Essential Q&A to Boost Query Performance

This article provides a comprehensive 15‑question guide on MySQL indexes, covering their definition, various types across data‑structure, physical and logical dimensions, situations where indexes fail or are unsuitable, B+‑tree advantages, search process, covering indexes, left‑most prefix, index push‑down, adding indexes to huge tables, using EXPLAIN to verify index usage, hash versus B+‑tree differences, pros and cons, and the distinction between clustered and non‑clustered indexes.

B+TreeIndexMySQL
0 likes · 18 min read
Mastering MySQL Indexes: 15 Essential Q&A to Boost Query Performance
Open Source Linux
Open Source Linux
May 5, 2023 · Databases

Mastering Subquery Unnesting: Turn Correlated Queries into Fast Joins

This article explains why correlated subqueries are a performance bottleneck, introduces the Apply (Correlated Join) operator and a systematic set of transformation rules that push Apply down and replace subqueries with efficient joins, covering scalar, existential, and quantified subqueries, aggregation handling, and set operations, all illustrated with TPC‑H examples.

Apply OperatorDatabase EnginesDecorrelate
0 likes · 17 min read
Mastering Subquery Unnesting: Turn Correlated Queries into Fast Joins
ITPUB
ITPUB
May 3, 2023 · Databases

Master MySQL Indexes: Types, Usage, and Optimization Tips

This comprehensive guide explains what MySQL indexes are, details various index types—including B+‑tree, hash, full‑text, and spatial—covers when indexes become ineffective, shows how to design, use, and troubleshoot them, and provides practical steps for large‑scale index management.

B+TreeDatabaseIndex
0 likes · 19 min read
Master MySQL Indexes: Types, Usage, and Optimization Tips
Liangxu Linux
Liangxu Linux
Apr 25, 2023 · Databases

How to Unnest Correlated Subqueries for Faster SQL Execution

This article explains why correlated subqueries are costly, introduces the Apply (correlated join) operator, and presents a series of systematic transformation rules that convert scalar, existential, and aggregate subqueries into efficient join‑based plans while preserving SQL semantics.

Apply OperatorDatabaseDecorrelation
0 likes · 17 min read
How to Unnest Correlated Subqueries for Faster SQL Execution
StarRocks
StarRocks
Apr 23, 2023 · Databases

Why Query Performance Optimization Matters and How to Master It

This guide explains the importance of query performance optimization for database products and engineers, outlines latency and throughput goals, shows how to locate bottlenecks with observability tools and Linux profilers, and provides practical high‑level and low‑level optimization techniques along with testing best practices.

CPU profilingQuery OptimizationStarRocks
0 likes · 16 min read
Why Query Performance Optimization Matters and How to Master It
ITPUB
ITPUB
Apr 18, 2023 · Databases

Mastering MySQL Index Merge: Principles, Algorithms, and Practical Tips

This article explains MySQL's index‑merge optimization, detailing how the EXPLAIN output indicates its use, the three underlying algorithms (intersect, union, sort‑union), practical query examples, configuration flags, and step‑by‑step guidance for testing with sample tables and indexes.

AlgorithmsDatabaseMySQL
0 likes · 9 min read
Mastering MySQL Index Merge: Principles, Algorithms, and Practical Tips
Sohu Tech Products
Sohu Tech Products
Mar 29, 2023 · Databases

Understanding the Purpose and Impact of WHERE 1=1 in SQL Queries

This article explains why developers often include the redundant condition “WHERE 1=1” in SQL statements, demonstrating through performance tests that it has no effect on execution time, and describing its practical use for simplifying the addition of further conditions in both static and dynamic queries.

Dynamic SQLQuery OptimizationSQL
0 likes · 6 min read
Understanding the Purpose and Impact of WHERE 1=1 in SQL Queries
ITPUB
ITPUB
Mar 25, 2023 · Big Data

Mastering Efficient SQL in ODPS: Union, Count‑Distinct, and Join Optimizations

This article walks through common SQL development scenarios on ODPS, examining why naïve UNION and COUNT DISTINCT can be slow, how to rewrite queries with GROUP BY, UNION ALL, JSON aggregation, and map‑join techniques, and shows the resulting execution‑plan improvements with concrete code and performance numbers.

Big DataCountDistinctMapJoin
0 likes · 17 min read
Mastering Efficient SQL in ODPS: Union, Count‑Distinct, and Join Optimizations
DataFunTalk
DataFunTalk
Mar 21, 2023 · Databases

Design and Technical Details of Apache Doris for Lakehouse Architecture

This article explains how Apache Doris extends its real‑time OLAP capabilities to support Lakehouse architectures, covering unified metadata, query acceleration, elastic compute, performance benchmarks, and future roadmap for richer data‑source integration and resource isolation.

Apache DorisBig DataLakehouse
0 likes · 20 min read
Design and Technical Details of Apache Doris for Lakehouse Architecture
Selected Java Interview Questions
Selected Java Interview Questions
Mar 12, 2023 · Big Data

Design and Optimization of Querying 100K Records from Tens of Millions of Data Using ClickHouse, Elasticsearch, HBase, and RediSearch

This article presents a comprehensive design and performance‑optimization study for extracting up to 100 000 records from a pool of tens of millions, comparing multithreaded ClickHouse pagination, Elasticsearch scroll‑scan, ES + HBase, and RediSearch + RedisJSON solutions, and provides practical recommendations based on measured latency and throughput.

ClickHouseHBaseQuery Optimization
0 likes · 11 min read
Design and Optimization of Querying 100K Records from Tens of Millions of Data Using ClickHouse, Elasticsearch, HBase, and RediSearch
Laravel Tech Community
Laravel Tech Community
Mar 6, 2023 · Databases

SQL Optimization Best Practices: Avoid SELECT *, Use Specific Fields, Index Strategies, and Other Performance Tips

This article presents a comprehensive set of MySQL performance guidelines, including avoiding SELECT *, preferring specific columns, eliminating OR in WHERE clauses, using numeric types, choosing VARCHAR over CHAR, applying proper JOINs, leveraging LIMIT, UNION ALL, composite indexes, and other practical tips to improve query efficiency and resource usage.

DatabaseIndexingMySQL
0 likes · 17 min read
SQL Optimization Best Practices: Avoid SELECT *, Use Specific Fields, Index Strategies, and Other Performance Tips