BestHub
Sign in
Tagged articles

Elasticsearch

1257 articles · Page 5 of 13
IT Services Circle
IT Services Circle
Mar 9, 2024 · Backend Development

Java Backend Interview Guide: Networks, Data Structures, MySQL Logs, Elasticsearch, Kafka, Microservices, and Spring Essentials

This article compiles a comprehensive Java backend interview guide covering network protocols, HTTP characteristics, core data structures and algorithms, MySQL logging mechanisms, Elasticsearch full‑text search, Kafka reliability and deduplication, common micro‑service components, load‑balancing algorithms, and essential Spring concepts such as IoC, AOP, transactions, and MVC workflow.

ElasticsearchInterviewJava
0 likes · 22 min read
Java Backend Interview Guide: Networks, Data Structures, MySQL Logs, Elasticsearch, Kafka, Microservices, and Spring Essentials
dbaplus Community
dbaplus Community
Mar 5, 2024 · Operations

How to Recover a Failing Elasticsearch Cluster: Master Loss, Shard Corruption, and More

This guide explains Elasticsearch cluster architecture, node roles, and metadata storage, then details step‑by‑step recovery procedures for master‑node loss, complete master outage, data‑node failures, shard allocation problems, corrupted shards, translog issues, and missing segment files, including relevant API commands and tool usage.

Cluster RecoveryData NodeElasticsearch
0 likes · 17 min read
How to Recover a Failing Elasticsearch Cluster: Master Loss, Shard Corruption, and More
Didi Tech
Didi Tech
Mar 5, 2024 · Databases

Migrating Didi's Log Retrieval from Elasticsearch to ClickHouse: Architecture, Challenges, and Performance Optimizations

Didi replaced its Elasticsearch‑based log platform with ClickHouse, redesigning architecture into isolated Log and Trace clusters, using hourly‑partitioned MergeTree tables and aggregating views to handle petabyte‑scale writes, diverse low‑latency queries, and high QPS, achieving over 400 nodes, 40 GB/s throughput, 30 % cost savings and four‑fold query latency reduction.

Big DataClickHouseData Storage
0 likes · 15 min read
Migrating Didi's Log Retrieval from Elasticsearch to ClickHouse: Architecture, Challenges, and Performance Optimizations
Alibaba Cloud Developer
Alibaba Cloud Developer
Mar 4, 2024 · Big Data

Elasticsearch Deep Dive: Features, Mapping & Zero‑Downtime Reindexing

This article provides a comprehensive overview of Elasticsearch, covering its distributed architecture, key features such as JSON RESTful APIs and multi‑tenant support, core functionalities like full‑text search and aggregations, comparisons with Solr, advanced mapping techniques, various query DSLs, suggestion mechanisms, and practical zero‑downtime reindexing strategies.

AggregationDSLElasticsearch
0 likes · 13 min read
Elasticsearch Deep Dive: Features, Mapping & Zero‑Downtime Reindexing
JD Tech
JD Tech
Feb 23, 2024 · Operations

Optimizing IT Ticket Management: Process Reengineering, Automation, Knowledge‑Base Pre‑position and Intelligent Q&A Integration

This article presents a comprehensive case study of how an IT operations team reduced weekly ticket volume from over 150 to fewer than 20 by classifying issues, redesigning workflows, introducing automation tools, deploying a knowledge‑base pre‑position strategy, and integrating an intelligent question‑answering chatbot.

ElasticsearchIT OperationsIntelligent QA
0 likes · 11 min read
Optimizing IT Ticket Management: Process Reengineering, Automation, Knowledge‑Base Pre‑position and Intelligent Q&A Integration
Code Ape Tech Column
Code Ape Tech Column
Feb 22, 2024 · Backend Development

Upgrading a Campus Blog Project to JDK 17 and Spring Boot 3: Process, Pitfalls, and Solutions

This article documents the step‑by‑step migration of a Java campus‑blog project from JDK 8 + Spring Boot 2.6 to JDK 17 and Spring Boot 3.2, explaining why the upgrade is worthwhile, detailing code and configuration changes, and sharing solutions to the numerous compatibility issues encountered.

ElasticsearchJavaSpringBoot3
0 likes · 18 min read
Upgrading a Campus Blog Project to JDK 17 and Spring Boot 3: Process, Pitfalls, and Solutions
Sohu Tech Products
Sohu Tech Products
Feb 21, 2024 · Backend Development

Sorting Image Filenames with Numbers in Elasticsearch: Script and Ingest‑Pipeline Solutions

The article explains how to sort image filenames containing numbers in Elasticsearch by either using a painless _script to extract the numeric part at query time or, more efficiently, by preprocessing filenames with an ingest pipeline that creates a numeric field for fast sorting, recommending the pipeline for performance‑critical use cases.

ElasticsearchGrokIngest Pipeline
0 likes · 8 min read
Sorting Image Filenames with Numbers in Elasticsearch: Script and Ingest‑Pipeline Solutions
NetEase Cloud Music Tech Team
NetEase Cloud Music Tech Team
Feb 21, 2024 · Artificial Intelligence

Cloud Music Public Opinion Analysis Platform: Architecture and GPT-Based Implementation

The article describes NetEase Cloud Music’s public‑opinion analysis platform, which integrates external and internal data streams into a layered architecture—ingestion, processing, storage in Elasticsearch, visualization, and monitoring—and employs GPT‑based analyzers for clustering, sentiment, summarization, and intelligent alerts while optimizing costs and planning automated GPT‑driven reports.

ElasticsearchGPT analysisSentiment Analysis
0 likes · 13 min read
Cloud Music Public Opinion Analysis Platform: Architecture and GPT-Based Implementation
The Dominant Programmer
The Dominant Programmer
Feb 20, 2024 · Backend Development

Integrating Elasticsearch with Spring Boot for Full CRUD Operations

This guide walks through integrating Spring Data Elasticsearch into a Spring Boot application, covering entity mapping annotations, repository creation, service layer implementation, and unit tests to perform create, read, update, delete, and fuzzy search operations on Elasticsearch indices.

CRUDElasticsearchJava
0 likes · 8 min read
Integrating Elasticsearch with Spring Boot for Full CRUD Operations
Architect's Guide
Architect's Guide
Feb 15, 2024 · Operations

Common ELK Deployment Architectures and Practical Solutions for Log Management

This article introduces the core components of the ELK stack, compares three typical deployment architectures—including Logstash‑only, Filebeat‑assisted, and Kafka‑backed designs—and provides concrete configuration examples and troubleshooting tips for multiline merging, timestamp handling, and module‑level log filtering.

ELKElasticsearchFilebeat
0 likes · 11 min read
Common ELK Deployment Architectures and Practical Solutions for Log Management
MaGe Linux Operations
MaGe Linux Operations
Feb 3, 2024 · Operations

How to Build a 100% High‑Availability Membership System with ES, Redis, and MySQL

This article details the design and implementation of a highly available membership system that handles billions of users and peak traffic of over 20,000 TPS, covering Elasticsearch dual‑center master‑slave clusters, traffic‑isolated three‑cluster architecture, Redis caching with distributed locks, and dual‑center MySQL partitioning.

ElasticsearchMySQLRedis
0 likes · 20 min read
How to Build a 100% High‑Availability Membership System with ES, Redis, and MySQL
Java Tech Enthusiast
Java Tech Enthusiast
Feb 2, 2024 · Backend Development

Boost Your Elasticsearch Development with Easy-Es: A Complete Guide

This article introduces Easy-Es, an ORM framework built on Elasticsearch's RestHighLevelClient, explains its architecture and advantages, and provides step‑by‑step instructions—including Maven/Gradle setup, configuration, entity and mapper creation, and full CRUD test examples—so developers can quickly integrate powerful search capabilities into Spring Boot applications.

Easy-EsElasticsearchJava
0 likes · 9 min read
Boost Your Elasticsearch Development with Easy-Es: A Complete Guide
Efficient Ops
Efficient Ops
Jan 31, 2024 · Databases

Why ClickHouse Beats Elasticsearch for High‑Performance Log Analytics

Facing data security and cost challenges in SaaS, the author evaluates ClickHouse versus Elasticsearch, highlighting ClickHouse’s superior write throughput, query speed, lower storage and CPU usage, and provides detailed deployment guides for Zookeeper, Kafka, FileBeat, and ClickHouse to build a cost‑effective private analytics platform.

Big DataClickHouseDatabase deployment
0 likes · 8 min read
Why ClickHouse Beats Elasticsearch for High‑Performance Log Analytics
macrozheng
macrozheng
Jan 29, 2024 · Backend Development

How Easy-Es Transforms Elasticsearch Operations in Java: A Step‑by‑Step Guide

This article introduces Easy-Es, an Elasticsearch ORM that mimics MyBatis‑Plus syntax, explains its key features, shows how to integrate it into a Spring Boot project, and provides comprehensive code examples for index management, CRUD, simple and advanced product searches, recommendation and aggregation functionalities.

Easy-EsElasticsearchJava
0 likes · 18 min read
How Easy-Es Transforms Elasticsearch Operations in Java: A Step‑by‑Step Guide
Architect
Architect
Jan 27, 2024 · Industry Insights

How We Built a Scalable Smart Customer Service System for an Activity Platform

This article details the end‑to‑end design, implementation, and operational results of a smart customer‑service platform that automates FAQ capture, leverages both Elasticsearch and LLM‑based models, and provides a low‑code, multi‑team backend for rapid issue resolution.

Elasticsearchfaq-automationlarge language model
0 likes · 13 min read
How We Built a Scalable Smart Customer Service System for an Activity Platform
Selected Java Interview Questions
Selected Java Interview Questions
Jan 25, 2024 · Backend Development

Easy-Es: An ORM Framework for Elasticsearch in Java

Easy-Es is a Java ORM framework built on Elasticsearch's RestHighLevelClient that simplifies search‑engine development, offers Mybatis‑Plus‑like usage, provides configuration and dependency details, and includes complete code examples for creating indexes and performing CRUD operations within Spring Boot applications.

Easy-EsElasticsearchJava
0 likes · 8 min read
Easy-Es: An ORM Framework for Elasticsearch in Java
dbaplus Community
dbaplus Community
Jan 21, 2024 · Databases

How to Use ES|QL in Elasticsearch 8.11: DSL vs SQL Queries Compared

This guide demonstrates how Elasticsearch 8.11's new ES|QL query language matches the traditional DSL syntax, covering basic searches, sorting, wildcard matching, aggregations, REST API nuances, and practical examples that show equivalent DSL and ES|QL statements with visual results.

AggregationDSLES|QL
0 likes · 6 min read
How to Use ES|QL in Elasticsearch 8.11: DSL vs SQL Queries Compared
Bilibili Tech
Bilibili Tech
Jan 16, 2024 · Artificial Intelligence

Design and Implementation of Bilibili's Intelligent Customer Service System

Bilibili created an AI‑powered customer‑service platform that integrates a WeChat Work chat UI, a conversation state machine, Elasticsearch and LLM‑based FAQ retrieval, and a low‑code admin console, replacing a thousand‑person manual support team, handling thousands of queries, boosting resolution rates and enabling cross‑team reuse.

ElasticsearchIntelligent Customer ServiceNLP
0 likes · 13 min read
Design and Implementation of Bilibili's Intelligent Customer Service System
Sanyou's Java Diary
Sanyou's Java Diary
Jan 11, 2024 · Backend Development

30 Essential Elasticsearch Tips to Boost Query Performance and Avoid Common Pitfalls

This article compiles practical Elasticsearch recommendations covering query caching, filter contexts, pagination, aggregation strategies, index mapping, shard design, and scripting best practices, providing developers with actionable insights to improve search performance, reduce resource consumption, and prevent common operational issues.

Elasticsearchquery-optimizationsearch engine
0 likes · 25 min read
30 Essential Elasticsearch Tips to Boost Query Performance and Avoid Common Pitfalls
JD Cloud Developers
JD Cloud Developers
Jan 10, 2024 · Artificial Intelligence

Boosting Elasticsearch with Generative AI: Relevance Engine & Vector Search

This article explores the rise of generative AI, outlines popular models like ChatGPT, DALL‑E, and Google Bard, examines their limitations, and then delves into Elasticsearch’s Relevance Engine and vector capabilities, demonstrating how to store, index, and query dense embeddings with practical code examples.

ElasticsearchLLM Integrationgenerative AI
0 likes · 17 min read
Boosting Elasticsearch with Generative AI: Relevance Engine & Vector Search
政采云技术
政采云技术
Jan 10, 2024 · Operations

Understanding and Improving Elasticsearch Shard Balancing Strategies

This article analyzes Elasticsearch shard imbalance incidents, explains the built‑in shard balancing algorithm and its configuration parameters, demonstrates weight calculations with source code, and proposes practical improvements—including shard count adjustments and a custom load‑aware balancing tool—to achieve more effective cluster load distribution.

Cluster OperationsElasticsearchload balancing
0 likes · 17 min read
Understanding and Improving Elasticsearch Shard Balancing Strategies
Java Architect Essentials
Java Architect Essentials
Jan 5, 2024 · Operations

Common ELK Deployment Architectures and Solutions for Log Management

This article introduces the ELK stack’s core components, compares four typical deployment architectures—including Logstash‑only, Filebeat‑based, and Kafka‑enhanced setups—discusses their trade‑offs, and provides practical configurations and solutions for multiline log merging, timestamp handling, and module‑specific filtering.

ELKElasticsearchFilebeat
0 likes · 11 min read
Common ELK Deployment Architectures and Solutions for Log Management
MaGe Linux Operations
MaGe Linux Operations
Jan 3, 2024 · Big Data

ClickHouse vs Elasticsearch: Faster, Cheaper Log Analytics Explained

This article compares ClickHouse and Elasticsearch for log analytics, highlighting ClickHouse's superior write throughput, query speed, and lower server costs, then provides a detailed, cost‑effective deployment guide covering Zookeeper, Kafka, FileBeat, ClickHouse installation, and visualization with ClickVisual, plus optimization tips.

Big DataClickHouseElasticsearch
0 likes · 15 min read
ClickHouse vs Elasticsearch: Faster, Cheaper Log Analytics Explained
Sohu Tech Products
Sohu Tech Products
Dec 27, 2023 · Operations

Why Does Elasticsearch Refresh Take 1‑5 Seconds? A Deep Dive into Index Settings and Soft Delete

This article records a systematic test of Elasticsearch refresh latency, revealing that update operations, a high proportion of deleted documents, and the soft‑delete setting significantly increase refresh time, while the large‑segment strategy and disabling soft delete can reduce latency without harming overall performance.

ElasticsearchSoft Deletebackend operations
0 likes · 7 min read
Why Does Elasticsearch Refresh Take 1‑5 Seconds? A Deep Dive into Index Settings and Soft Delete
Efficient Ops
Efficient Ops
Dec 27, 2023 · Big Data

Why ClickHouse Beats Elasticsearch for Log Analytics – Performance, Cost & Deployment

This article compares ClickHouse and Elasticsearch for log analytics, highlighting ClickHouse’s superior write throughput, query speed, and lower server costs, then details a cost‑effective deployment architecture—including Zookeeper, Kafka, FileBeat, and ClickHouse setup—and shares optimization tips and visualization using ClickVisual.

Big DataClickHouseElasticsearch
0 likes · 13 min read
Why ClickHouse Beats Elasticsearch for Log Analytics – Performance, Cost & Deployment
Weimob Technology Center
Weimob Technology Center
Dec 22, 2023 · Big Data

Unlocking Elasticsearch at Scale: Real‑World Practices from Weimob

The Weimob Technology Salon session on "Elasticsearch in Weimob's Practice" shares practical usage recommendations, monitoring setups with Prometheus and Grafana, field‑type guidance, and solutions to common operational challenges, offering developers actionable insights for high‑performance search deployments.

Big DataElasticsearchMonitoring
0 likes · 5 min read
Unlocking Elasticsearch at Scale: Real‑World Practices from Weimob
DeWu Technology
DeWu Technology
Dec 18, 2023 · Databases

Elasticsearch Best Practices: Query, Index, and Performance Optimizations

The guide outlines production‑ready Elasticsearch best practices, covering query tuning such as using shard request cache, filter context, size‑0 aggregations and composite aggregations; write strategies like auto‑generated IDs, bulk API sizing and refresh handling; optimal shard counts, explicit mappings with disabled unnecessary features, and general advice to use explicit index names and stored scripts.

ElasticsearchIndexingPerformance
0 likes · 22 min read
Elasticsearch Best Practices: Query, Index, and Performance Optimizations
macrozheng
macrozheng
Dec 12, 2023 · Backend Development

RediSearch vs Elasticsearch: Cheap Setup, Performance Benchmarks, Java Guide

RediSearch, a low‑memory Redis module, offers comparable full‑text search capabilities to Elasticsearch, with faster indexing and query performance on modest hardware; this article explains its features, compares benchmarks, shows Docker installation, and provides Java/Jedis code examples for creating, querying, and managing indexes.

DockerElasticsearchFull-text Search
0 likes · 11 min read
RediSearch vs Elasticsearch: Cheap Setup, Performance Benchmarks, Java Guide
Code Ape Tech Column
Code Ape Tech Column
Dec 12, 2023 · Operations

Centralized Log Collection with Filebeat and Graylog

This article explains how to use Filebeat together with Graylog to collect, ship, store, and analyze logs from multiple environments, covering tool introductions, configuration files, Docker deployment, Spring Boot integration, and practical search syntax for effective log monitoring.

ElasticsearchFilebeatGraylog
0 likes · 20 min read
Centralized Log Collection with Filebeat and Graylog
Su San Talks Tech
Su San Talks Tech
Dec 9, 2023 · Databases

Boost Large Table Reads with Query Separation

This article explains the concept of query separation, its suitable scenarios, implementation methods—including synchronous, asynchronous, and binlog approaches—storage options, MQ considerations, and how it can dramatically improve query performance for massive tables.

AsynchronousElasticsearchMQ
0 likes · 9 min read
Boost Large Table Reads with Query Separation
MaGe Linux Operations
MaGe Linux Operations
Nov 28, 2023 · Databases

When to Use Synonyms in Elasticsearch: Index-Time vs Search-Time

This article explains how Elasticsearch tokenizes text, why small spelling errors or plural forms can miss matches, and how synonyms improve search recall, comparing the trade‑offs of applying synonyms during indexing versus at query time, with practical code examples and tips for managing large synonym lists.

AnalysisElasticsearchIndexing
0 likes · 7 min read
When to Use Synonyms in Elasticsearch: Index-Time vs Search-Time
vivo Internet Technology
vivo Internet Technology
Nov 22, 2023 · Operations

Investigation and Resolution of Elasticsearch node_concurrent_recoveries Performance Issue

The team traced read‑request timeouts to a single overloaded Elasticsearch node where an excessively high node_concurrent_recoveries setting caused many simultaneous shard recoveries and disk‑watermark‑driven relocations, and resolved the issue by lowering concurrent recoveries, enabling adaptive replica selection, and adjusting allocation settings.

CPUClusterElasticsearch
0 likes · 16 min read
Investigation and Resolution of Elasticsearch node_concurrent_recoveries Performance Issue
Top Architecture Tech Stack
Top Architecture Tech Stack
Nov 9, 2023 · Big Data

Full-Text Search Overview and Elasticsearch Introduction with Installation Guide

This article explains the concept of full-text retrieval, introduces Elasticsearch as a popular open‑source search engine built on Apache Lucene, and provides detailed step‑by‑step installation instructions for both traditional setups and Docker containers, including required environment configuration and common troubleshooting.

ElasticsearchFull-text Searchsearch engine
0 likes · 6 min read
Full-Text Search Overview and Elasticsearch Introduction with Installation Guide
Baidu Geek Talk
Baidu Geek Talk
Nov 8, 2023 · Databases

BES Engineering Practices for Large‑Scale Vector Database Scenarios

At QCon 2023, Baidu’s BES team detailed how their cloud‑native Elasticsearch service has been engineered for large‑scale vector search, describing architecture, C++ plugin integration, memory‑saving storage tricks, HNSW/IVF optimizations, filter strategies, and real‑world multimodal video and LLM knowledge‑base deployments.

AIBESElasticsearch
0 likes · 16 min read
BES Engineering Practices for Large‑Scale Vector Database Scenarios
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
Nov 1, 2023 · Databases

How BES Powers Large-Scale Vector Search for AI Applications

This article explains the principles of vector databases, outlines the engineering practices of Baidu Intelligent Cloud BES for large‑scale vector retrieval, discusses optimization techniques such as HNSW, IVF and filter integration, and presents real‑world AI use cases and future development directions.

AIBESElasticsearch
0 likes · 16 min read
How BES Powers Large-Scale Vector Search for AI Applications
HelloTech
HelloTech
Oct 31, 2023 · Big Data

Investigation of Data Loss in a Flink Kafka Consumer Caused by Mixed Kafka Cluster Configuration

The data loss in a Flink‑Kafka job was caused by a mis‑configured bootstrap.servers list that mixed production and pre‑release Kafka clusters, leading different subtasks to connect to different clusters, resulting in inconsistent partition discovery and offset fetching, which omitted several partitions until the list was corrected.

Cluster ConfigurationData lossElasticsearch
0 likes · 8 min read
Investigation of Data Loss in a Flink Kafka Consumer Caused by Mixed Kafka Cluster Configuration
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Oct 30, 2023 · Databases

Easy-Es Tutorial: Simplify Elasticsearch Operations with a Java ORM Framework

This article introduces Easy-Es, a Java ORM framework built on Elasticsearch's RestHighLevelClient that simplifies query construction and CRUD operations, provides step‑by‑step setup including Maven dependencies, configuration, entity and mapper definitions, controller examples, and advanced features such as condition builders, index management, logging, and aggregation queries.

AggregationCRUDEasy-Es
0 likes · 8 min read
Easy-Es Tutorial: Simplify Elasticsearch Operations with a Java ORM Framework
Code Ape Tech Column
Code Ape Tech Column
Oct 24, 2023 · Big Data

Synchronizing MySQL Data to Elasticsearch Using Logstash

This tutorial explains how to set up the environment, configure Elasticsearch and Logstash, create the necessary MySQL tables, and use a Logstash pipeline to continuously sync MySQL records into an Elasticsearch index, while also covering common pitfalls and troubleshooting steps.

Data synchronizationETLElasticsearch
0 likes · 12 min read
Synchronizing MySQL Data to Elasticsearch Using Logstash
Liangxu Linux
Liangxu Linux
Oct 10, 2023 · Operations

Master Kibana: Install, Configure, and Visualize Elasticsearch Data Step‑by‑Step

This guide walks you through installing Kibana, configuring its connection to Elasticsearch, creating index patterns, using Discover for searches, mastering Lucene‑based query syntax, building visualizations, assembling dashboards, and monitoring logs, all illustrated with clear screenshots and code examples.

DashboardElasticsearchKibana
0 likes · 14 min read
Master Kibana: Install, Configure, and Visualize Elasticsearch Data Step‑by‑Step
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Oct 10, 2023 · Backend Development

Master Elasticsearch with Spring Boot: 14 Essential Java API Operations

Learn how to integrate Elasticsearch 7.8 with Spring Boot 2.4, covering essential dependencies, index management, document CRUD, bulk operations, and advanced queries such as pagination, sorting, filtering, highlighting, aggregations, and grouping, all demonstrated with concise Java High-Level REST client code examples.

ElasticsearchJavaREST API
0 likes · 14 min read
Master Elasticsearch with Spring Boot: 14 Essential Java API Operations
ITPUB
ITPUB
Oct 4, 2023 · Backend Development

How to Speed Up Slow Elasticsearch Aggregations with execution_hint "map"

In a high‑traffic e‑commerce system, sharding makes cross‑shop queries inefficient, and adding terms aggregations in Elasticsearch caused queries to take dozens of seconds, but using the "execution_hint":"map" option dramatically reduces aggregation latency.

AggregationBig DataElasticsearch
0 likes · 7 min read
How to Speed Up Slow Elasticsearch Aggregations with execution_hint "map"
Code Ape Tech Column
Code Ape Tech Column
Sep 26, 2023 · Databases

Query Separation: Optimizing Large Table Reads by Decoupling Query Data

The article explains query separation as a technique to improve performance of massive business tables by storing write‑optimized data and query‑optimized data separately, discusses when to apply it, various implementation methods, storage options, and operational considerations such as MQ handling and consistency.

Asynchronous ReplicationElasticsearchcold hot data
0 likes · 10 min read
Query Separation: Optimizing Large Table Reads by Decoupling Query Data
Architect
Architect
Sep 19, 2023 · Big Data

How Tianyan Beats ELK: Inside a High‑Performance Distributed Log Service

This article analyzes the challenges of logging in distributed services, compares the traditional ELK stack with Baidu's Tianyan platform, and details Tianyan's architecture, data collection, high‑throughput transmission, storage, retrieval, resource isolation, dynamic cleanup, and best‑practice recommendations, complete with code examples and performance insights.

Big DataELKElasticsearch
0 likes · 30 min read
How Tianyan Beats ELK: Inside a High‑Performance Distributed Log Service
Top Architect
Top Architect
Sep 19, 2023 · Operations

Building a Real‑Time ELK Log Analysis Platform and Integrating It with Spring Boot and Nginx

This tutorial explains why centralized log collection is essential for micro‑service systems, introduces the ELK stack (Elasticsearch, Logstash, Kibana), provides step‑by‑step installation on Ubuntu, shows how to configure Logstash shipper and indexer pipelines, and demonstrates integration with Spring Boot and Nginx logs for real‑time monitoring.

ELKElasticsearchKibana
0 likes · 20 min read
Building a Real‑Time ELK Log Analysis Platform and Integrating It with Spring Boot and Nginx
HelloTech
HelloTech
Sep 19, 2023 · Databases

Deep Dive into Elasticsearch RestClient Sniffer and Node Discovery Mechanism

The article explains how our driver‑passenger matching service migrated from load‑balanced Elasticsearch access to a direct RestClient, then automated node discovery using the built‑in Sniffer and SniffOnFailureListener, detailing its scheduling, request logic, and how this eliminates manual IP management while keeping the client in sync with cluster topology.

ElasticsearchJavaNode Discovery
0 likes · 11 min read
Deep Dive into Elasticsearch RestClient Sniffer and Node Discovery Mechanism
Su San Talks Tech
Su San Talks Tech
Sep 9, 2023 · Backend Development

How to Sync MySQL Binlog to Elasticsearch Using Canal and RocketMQ

This step‑by‑step tutorial shows how to configure Alibaba's Canal to capture MySQL binlog changes, route them through RocketMQ, and index the data into Elasticsearch, covering cluster mode, MySQL and Elasticsearch setup, RocketMQ topic creation, Canal properties, and consumer implementation.

CanalData synchronizationElasticsearch
0 likes · 8 min read
How to Sync MySQL Binlog to Elasticsearch Using Canal and RocketMQ
JD Retail Technology
JD Retail Technology
Sep 4, 2023 · Big Data

JD Mini Program Data Center: Architecture, Milestones, and Real‑time Analytics Solutions

The article details the JD Mini Program platform, its data‑center development milestones, comprehensive business panorama, technical architecture, data collection, storage, and analysis pipelines—including Flink‑based real‑time monitoring, ClickHouse custom analytics, and Elasticsearch user‑behavior insights—while outlining current challenges and future AI‑driven enhancements.

Big DataClickHouseElasticsearch
0 likes · 16 min read
JD Mini Program Data Center: Architecture, Milestones, and Real‑time Analytics Solutions
Shepherd Advanced Notes
Shepherd Advanced Notes
Aug 30, 2023 · Databases

Achieving Efficient Real‑Time Search for Massive Data in Spring Boot Applications

The article analyzes why massive tables become a bottleneck in Spring Boot systems, outlines the drawbacks of sharding, and presents a layered solution—data archiving, read‑write separation with caching, heterogeneous source synchronization via Elasticsearch and Canal, and selective sharding—to enable high‑performance real‑time search.

CanalData ArchivingElasticsearch
0 likes · 13 min read
Achieving Efficient Real‑Time Search for Massive Data in Spring Boot Applications
Efficient Ops
Efficient Ops
Aug 28, 2023 · Operations

How to Build a Real‑Time ELK Log Analysis Platform for Scalable Operations

This article explains why centralized logging is essential for modern micro‑service systems, outlines an ELK‑based architecture with Filebeat, Kafka, Logstash, Elasticsearch and Kibana, and provides detailed configuration examples for both file‑based and Kubernetes‑based log collection, plus visualization techniques.

ELKElasticsearchFilebeat
0 likes · 10 min read
How to Build a Real‑Time ELK Log Analysis Platform for Scalable Operations
iQIYI Technical Product Team
iQIYI Technical Product Team
Aug 25, 2023 · Big Data

Venus Log Platform Architecture Evolution: From ELK to Data Lake

The Venus log platform at iQiyi migrated from an ElasticSearch‑Kibana architecture to an Iceberg‑based data lake with Trino, cutting storage and compute costs by over 70%, boosting stability by 85%, and efficiently supporting billions of daily logs through write‑heavy, low‑query workloads.

Big DataElasticsearchIceberg
0 likes · 22 min read
Venus Log Platform Architecture Evolution: From ELK to Data Lake
政采云技术
政采云技术
Aug 17, 2023 · Backend Development

Traffic‑Based Quality Assurance Framework for Elasticsearch Search Service

This article presents a traffic‑driven quality assurance framework for an Elasticsearch‑based search service, detailing active and inactive code protection strategies, automated scenario generation from Dubbo logs, template fingerprinting, de‑duplication, expected result pools, and validation rules to ensure comprehensive test coverage and reliable regression.

Elasticsearchautomated testingquality assurance
0 likes · 13 min read
Traffic‑Based Quality Assurance Framework for Elasticsearch Search Service
Code Ape Tech Column
Code Ape Tech Column
Aug 15, 2023 · Operations

High‑Availability Architecture for a Billion‑Scale Membership System: Dual‑Center ES, Redis, and MySQL Solutions

This article details the design and implementation of a highly available, high‑performance membership system serving over a billion users, covering dual‑center Elasticsearch clusters, traffic‑isolated three‑cluster ES architecture, Redis dual‑center caching, MySQL partitioned clusters, migration strategies, and refined flow‑control and degradation mechanisms.

ElasticsearchMySQLScaling
0 likes · 20 min read
High‑Availability Architecture for a Billion‑Scale Membership System: Dual‑Center ES, Redis, and MySQL Solutions
Code Ape Tech Column
Code Ape Tech Column
Aug 11, 2023 · Big Data

Elasticsearch Pagination: From/Size, Deep Paging Issues, and Alternative Methods (Scroll, Search After, PIT)

This article explains how Elasticsearch pagination works with from/size, why deep paging can cause performance problems, and compares alternative techniques such as Scroll, Scroll‑Scan, Sliced Scroll, Search After, and point‑in‑time (PIT) searches for handling large result sets efficiently.

Deep PagingElasticsearchPagination
0 likes · 17 min read
Elasticsearch Pagination: From/Size, Deep Paging Issues, and Alternative Methods (Scroll, Search After, PIT)
Didi Tech
Didi Tech
Aug 10, 2023 · Information Security

Security Hardening and Architecture of Didi's Elasticsearch Deployment

Didi hardened its massive Elasticsearch deployment—spanning 66 clusters and thousands of nodes—by adding a custom security plugin that authenticates requests at the cluster level, implementing a one‑click toggle and staged rolling upgrades, ultimately enabling authentication across all clusters and dramatically reducing data‑leak risk.

DidiElasticsearchPlugin
0 likes · 12 min read
Security Hardening and Architecture of Didi's Elasticsearch Deployment
Didi Tech
Didi Tech
Aug 9, 2023 · Backend Development

Upgrading Didi Elasticsearch to JDK 17 with ZGC: Challenges, Solutions, and Performance Gains

Didi upgraded its self‑developed Elasticsearch from JDK 11/G1 to JDK 17, adopting ZGC for latency‑critical clusters and tuned G1 for throughput, which eliminated long GC pauses, reduced query latency by up to 96%, cut CPU usage, and dramatically improved stability across multiple production clusters.

ElasticsearchGC optimizationMonitoring
0 likes · 14 min read
Upgrading Didi Elasticsearch to JDK 17 with ZGC: Challenges, Solutions, and Performance Gains
ITPUB
ITPUB
Aug 9, 2023 · Operations

Why Is My Elasticsearch Cluster Using 15 GB Heap? A Deep Dive into Memory Bottlenecks

The article examines a 7‑node Elasticsearch cluster with 500 million documents, uncovering excessive heap usage, high OS memory pressure, numerous deleted documents, large translog, low query‑cache hit rate, and an over‑sharded design, then offers concrete tuning and redesign recommendations to restore performance.

Cluster OperationsElasticsearchheap usage
0 likes · 16 min read
Why Is My Elasticsearch Cluster Using 15 GB Heap? A Deep Dive into Memory Bottlenecks
Java Interview Crash Guide
Java Interview Crash Guide
Aug 8, 2023 · Operations

How We Built 99.99% High Availability for a Billion‑User Membership System

This article details the end‑to‑end high‑availability architecture—including dual‑center Elasticsearch clusters, Redis caching with distributed locks, and a dual‑center MySQL partitioned setup—that enables a membership platform serving billions of users to sustain massive traffic while ensuring data consistency and rapid recovery.

ElasticsearchMySQLRedis
0 likes · 21 min read
How We Built 99.99% High Availability for a Billion‑User Membership System
Didi Tech
Didi Tech
Aug 7, 2023 · Backend Development

How Didi Achieved Cross‑Datacenter Elasticsearch Replication for Strong Consistency

This article explains Didi's self‑developed DCDR system that replicates Elasticsearch indices across data‑center clusters, detailing its design goals, core mechanisms, chain construction, historical data recovery, real‑time sync, and data‑quality validation to ensure high availability and strong consistency.

Cross‑Datacenter ReplicationDCDRData Consistency
0 likes · 15 min read
How Didi Achieved Cross‑Datacenter Elasticsearch Replication for Strong Consistency
MaGe Linux Operations
MaGe Linux Operations
Aug 5, 2023 · Databases

Elasticsearch vs ClickHouse: Architecture, Queries, and Performance

This article compares Elasticsearch and ClickHouse by examining their underlying architectures, node roles, query languages, and performance through a series of benchmark tests using Docker‑compose, Vector data pipelines, and Python SDKs, revealing ClickHouse’s superior speed in most query scenarios despite lacking advanced search features.

ClickHouseElasticsearch
0 likes · 12 min read
Elasticsearch vs ClickHouse: Architecture, Queries, and Performance
Efficient Ops
Efficient Ops
Aug 2, 2023 · Databases

Why ClickHouse Outperforms Elasticsearch in Real‑World Queries

This article compares Elasticsearch and ClickHouse across architecture, query capabilities, and performance using Docker‑compose stacks and Python SDK tests, demonstrating that ClickHouse often delivers superior speed, especially in aggregation and regex queries, while highlighting each system’s design trade‑offs.

ClickHouseDatabase ComparisonDocker Compose
0 likes · 13 min read
Why ClickHouse Outperforms Elasticsearch in Real‑World Queries
Architect
Architect
Jul 28, 2023 · Databases

Which MySQL‑to‑Elasticsearch Sync Method Wins? 4 Solutions & Tool Picks

The article compares four MySQL‑to‑Elasticsearch synchronization approaches—synchronous dual‑write, asynchronous dual‑write via MQ, timer‑based SQL extraction, and binlog‑based real‑time sync—evaluates their trade‑offs, and recommends practical tools such as Canal, DTS, and Databus for implementation.

CanalDTSDatabus
0 likes · 13 min read
Which MySQL‑to‑Elasticsearch Sync Method Wins? 4 Solutions & Tool Picks
Top Architect
Top Architect
Jul 27, 2023 · Big Data

Performance Comparison of Elasticsearch and ClickHouse for Log Search

This article compares Elasticsearch and ClickHouse as log‑search solutions, detailing their architectures, Docker‑compose deployments, data‑ingestion pipelines with Vector, query syntax differences, and benchmark results that show ClickHouse generally outperforms Elasticsearch in speed and aggregation efficiency.

Big DataClickHouseDocker
0 likes · 13 min read
Performance Comparison of Elasticsearch and ClickHouse for Log Search
SQB Blog
SQB Blog
Jul 20, 2023 · Artificial Intelligence

How We Built and Optimized a Multi‑Pool Recommendation System for Boss Circle

This article explains the design, implementation, and iterative optimization of Boss Circle's recommendation engine, covering the initial simple ranking, the introduction of Elasticsearch‑based scoring, multi‑pool data sources, machine‑learning experiments, real‑time feature handling, and future personalization challenges.

Data pipelinesElasticsearchpersonalization
0 likes · 17 min read
How We Built and Optimized a Multi‑Pool Recommendation System for Boss Circle
Top Architect
Top Architect
Jul 18, 2023 · Fundamentals

Comprehensive Introduction to Elasticsearch: Core Concepts, Architecture, and Practical Usage

This article provides a detailed overview of Elasticsearch, covering its underlying Lucene technology, data types, indexing mechanisms, cluster architecture, shard and replica management, mapping definitions, installation steps, health monitoring, write and storage processes, and performance optimization techniques for production deployments.

ElasticsearchIndexingLucene
0 likes · 36 min read
Comprehensive Introduction to Elasticsearch: Core Concepts, Architecture, and Practical Usage
Architect
Architect
Jul 17, 2023 · Databases

Performance Comparison of Elasticsearch and ClickHouse for Log Search and Analytics

This article compares Elasticsearch and ClickHouse by describing their architectures, presenting Docker‑based test stacks, showing code snippets for deployment, data ingestion, and queries, and reporting performance results that demonstrate ClickHouse generally outperforms Elasticsearch in log‑analytics scenarios.

ClickHouseDockerElasticsearch
0 likes · 12 min read
Performance Comparison of Elasticsearch and ClickHouse for Log Search and Analytics
Efficient Ops
Efficient Ops
Jul 16, 2023 · Operations

Mastering ELK: Deploy Architectures, Multiline Logs, and Kibana Tips

This guide explains the three main ELK deployment architectures, compares Logstash and Filebeat collectors, introduces a cache‑queue option for high‑volume logs, and provides practical solutions for multiline log merging, timestamp correction, and module‑level filtering in Kibana, helping operations teams build efficient log pipelines.

ELKElasticsearchFilebeat
0 likes · 10 min read
Mastering ELK: Deploy Architectures, Multiline Logs, and Kibana Tips
Programmer DD
Programmer DD
Jul 15, 2023 · Operations

Why Is My Elasticsearch Cluster Using 99% Memory? Sharding, Translog & JVM Insights

This article analyzes a 7‑node Elasticsearch cluster with 500 million documents, revealing excessive shard count, high heap and OS memory usage, large translog, low query‑cache hit rate, and heavy I/O, and offers concrete recommendations on mapping, sharding, JVM tuning, and resource management to restore performance.

Cluster MonitoringElasticsearchJVM
0 likes · 17 min read
Why Is My Elasticsearch Cluster Using 99% Memory? Sharding, Translog & JVM Insights
Selected Java Interview Questions
Selected Java Interview Questions
Jul 15, 2023 · Operations

High‑Availability Architecture for a Large‑Scale Membership System

The article describes how a membership system serving billions of users across multiple platforms achieves high performance and high availability through dual‑center Elasticsearch clusters, traffic‑isolated three‑cluster ES architecture, Redis caching with distributed locks, dual‑center MySQL partitioning, and fine‑grained flow‑control and degradation strategies.

ElasticsearchMySQLRedis
0 likes · 25 min read
High‑Availability Architecture for a Large‑Scale Membership System
ITPUB
ITPUB
Jul 12, 2023 · Databases

How We Migrated a Multi‑Petabyte Elasticsearch Cluster Across Data Centers Without Downtime

This article details the end‑to‑end process of moving Qunar's massive Elasticsearch logging cluster from a saturated data‑center to a new facility, covering background constraints, migration planning, manual and automated steps, performance‑tuning parameters, shard‑balancing techniques, and the final outcomes achieved.

Cluster MigrationData CenterElasticsearch
0 likes · 21 min read
How We Migrated a Multi‑Petabyte Elasticsearch Cluster Across Data Centers Without Downtime