BestHub
Sign in
Tagged articles

Elasticsearch

1286 articles · Page 10 of 13
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 13, 2020 · Big Data

Elasticsearch Write, Read, Search Processes and Performance Tuning Guide

This article explains Elasticsearch's data ingestion, retrieval, and search workflows, details the underlying indexing mechanisms, and provides comprehensive system‑level, shard‑level, and query‑level tuning recommendations—including configuration snippets and best‑practice strategies for high‑throughput and low‑latency deployments.

Cluster ConfigurationElasticsearchindexing
0 likes · 20 min read
Elasticsearch Write, Read, Search Processes and Performance Tuning Guide
DeWu Technology
DeWu Technology
Dec 11, 2020 · Big Data

Data Synchronization from MySQL to Elasticsearch using DataX and Canal

The article explains how to improve query performance by flattening multi‑table MySQL data and synchronizing it to Elasticsearch—using DataX for one‑time bulk loading and Canal (with Canal‑Adapter) for real‑time binlog‑driven incremental updates—while detailing configuration steps, job examples, and common pitfalls.

CanalDataXETL
0 likes · 14 min read
Data Synchronization from MySQL to Elasticsearch using DataX and Canal
Programmer DD
Programmer DD
Dec 9, 2020 · Operations

Step-by-Step Guide to Installing Apache SkyWalking with Elasticsearch and InfluxDB

This tutorial walks through installing and configuring Apache SkyWalking, an open‑source APM system for micro‑services and cloud‑native environments, covering its architecture, Elasticsearch and InfluxDB storage setup, agent deployment, service startup, alarm integration, and essential documentation links.

APMDockerElasticsearch
0 likes · 12 min read
Step-by-Step Guide to Installing Apache SkyWalking with Elasticsearch and InfluxDB
Tencent Cloud Developer
Tencent Cloud Developer
Dec 7, 2020 · Big Data

Searchable Snapshots in Elasticsearch 7.10: Features, Usage, and Future Outlook

Elasticsearch 7.10 adds searchable snapshots, letting users query indices stored directly in remote repositories such as S3 or COS, which halves storage costs, decouples storage from compute, supports manual mounting and ILM cold‑phase policies, and promises future full storage‑compute separation without local caching.

ElasticsearchILMSearchable Snapshots
0 likes · 12 min read
Searchable Snapshots in Elasticsearch 7.10: Features, Usage, and Future Outlook
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 2, 2020 · Big Data

Elasticsearch Search Request Structure and Query DSL Guide

This article provides a comprehensive guide to Elasticsearch search requests, detailing the routing process, request structure, core modules like query, size, from, _source, and sort, and illustrating various query and filter types such as match, term, range, bool, and wildcard with practical curl examples.

ElasticsearchFilteringquery-dsl
0 likes · 21 min read
Elasticsearch Search Request Structure and Query DSL Guide
System Architect Go
System Architect Go
Nov 30, 2020 · Databases

Five Ways to Sync MySQL Data to Elasticsearch, Redis, MQ, etc.

This article outlines five practical methods for synchronizing MySQL data to external systems such as Elasticsearch, Redis, and message queues, covering business‑layer hooks, middleware integration, scheduled tasks using updated_at, binlog parsing with ROW format, and handling mixed or statement binlog formats, plus open‑source tools.

BinlogElasticsearchKafka
0 likes · 5 min read
Five Ways to Sync MySQL Data to Elasticsearch, Redis, MQ, etc.
vivo Internet Technology
vivo Internet Technology
Nov 25, 2020 · Databases

Understanding Elasticsearch Architecture, Indexing, and Storage Mechanisms

Elasticsearch combines Lucene’s inverted index with a distributed cluster of master‑eligible, data, and coordinating nodes, using Zen discovery for node election and split‑brain prevention, while indexing writes to primary shards, replicating to replicas, storing immutable segments that are periodically merged for efficient search.

ClusterDistributed ArchitectureElasticsearch
0 likes · 19 min read
Understanding Elasticsearch Architecture, Indexing, and Storage Mechanisms
DevOps Cloud Academy
DevOps Cloud Academy
Nov 24, 2020 · Operations

Deploying the EFK Stack with Local‑Volume StorageClass on OpenShift

This guide explains how to prepare resources, create a local‑volume storage class, install the Elasticsearch and Cluster Logging operators, and configure a persistent EFK stack on OpenShift, including YAML definitions, command‑line steps, and best‑practice notes on node selectors and tolerations.

EFKElasticsearchLocal Volume
0 likes · 12 min read
Deploying the EFK Stack with Local‑Volume StorageClass on OpenShift
DevOps Cloud Academy
DevOps Cloud Academy
Nov 24, 2020 · Databases

Elasticsearch Interview Questions and Answers: Fundamentals, Architecture, and Operations

This article provides a detailed collection of basic Elasticsearch interview questions and answers, covering core concepts such as cluster and node architecture, indexing, mapping, search APIs, analyzers, tokenizers, aggregation, security, and operational tools, along with practical code snippets for common tasks.

AggregationClusterElasticsearch
0 likes · 19 min read
Elasticsearch Interview Questions and Answers: Fundamentals, Architecture, and Operations
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Nov 22, 2020 · Databases

Various Data Synchronization Architectures for Real-Time Elasticsearch Integration

The article compares five data synchronization approaches—periodic Logstash pulls, synchronous dual writes, asynchronous dual writes with MQ, Canal-based binlog streaming, and a Canal‑MQ hybrid—detailing their architectures, advantages, drawbacks, and suitable scenarios for integrating databases with Elasticsearch.

CanalElasticsearchLogstash
0 likes · 4 min read
Various Data Synchronization Architectures for Real-Time Elasticsearch Integration
dbaplus Community
dbaplus Community
Nov 19, 2020 · Big Data

How Banks Can Tame Petabytes of Unstructured Data: Architecture and Best Practices

This article presents a comprehensive design and deployment plan for a bank's unstructured data service platform, covering data growth challenges, lifecycle management, three‑tier storage architecture, Elasticsearch indexing, fault‑tolerant disaster recovery, monitoring, and future development directions.

Elasticsearchdisaster recoverystorage architecture
0 likes · 19 min read
How Banks Can Tame Petabytes of Unstructured Data: Architecture and Best Practices
MaGe Linux Operations
MaGe Linux Operations
Nov 19, 2020 · Backend Development

Supercharging Elasticsearch: Practical Index & Search Optimizations for Billion-Row Queries

This article shares practical Elasticsearch and Lucene optimization techniques—including index structure tuning, shard routing, DocValues management, and query pagination—to achieve sub‑second search performance on datasets exceeding a billion records while supporting multi‑year historical queries.

ElasticsearchLuceneindexing
0 likes · 13 min read
Supercharging Elasticsearch: Practical Index & Search Optimizations for Billion-Row Queries
System Architect Go
System Architect Go
Nov 17, 2020 · Big Data

Elasticsearch Distributed Search Mechanisms: query_then_fetch and dfs_query_then_fetch

Elasticsearch provides two search types—query_then_fetch (default) and dfs_query_then_fetch—each involving a multi-step process where the client node distributes queries to relevant shards, shards execute searches using local or global term frequencies, aggregate results, and retrieve full documents, with noted trade‑offs.

ElasticsearchQuery_then_fetchdfs_query_then_fetch
0 likes · 5 min read
Elasticsearch Distributed Search Mechanisms: query_then_fetch and dfs_query_then_fetch
System Architect Go
System Architect Go
Nov 16, 2020 · Backend Development

How to Decouple Elasticsearch Queries Using Mustache Search Templates

This article explains how to separate Elasticsearch DSL queries from application code by defining reusable search templates with Mustache placeholders, managing them via the scripts API, and rendering them with parameters to produce flexible, maintainable search requests.

BackendDSLElasticsearch
0 likes · 10 min read
How to Decouple Elasticsearch Queries Using Mustache Search Templates
Wukong Talks Architecture
Wukong Talks Architecture
Nov 13, 2020 · Cloud Native

Understanding Distributed Tracing with Spring Cloud Sleuth and Zipkin

This article explains the principles and practical steps for implementing distributed tracing in microservice architectures using Spring Cloud Sleuth and Zipkin, covering why tracing is needed, core concepts like Span and Trace, Maven integration, Docker deployment, Zipkin UI usage, and performance analysis.

DockerElasticsearchJava
0 likes · 13 min read
Understanding Distributed Tracing with Spring Cloud Sleuth and Zipkin
Tencent Cloud Developer
Tencent Cloud Developer
Nov 10, 2020 · Big Data

Design and Optimization of a Real-Time Video Recommendation Indexing System

The article describes a real‑time video recommendation indexing system that replaces 30‑minute batch builds with an Elasticsearch‑based service, integrates prior and posterior data pipelines, ensures consistency via locking and version checks, enables zero‑downtime upgrades, smooths write spikes, and boosts recall performance through multi‑level caching and ES tuning, delivering sub‑40 ms latency and significant business growth.

CachingElasticsearchFlink
0 likes · 13 min read
Design and Optimization of a Real-Time Video Recommendation Indexing System
DevOps Cloud Academy
DevOps Cloud Academy
Nov 7, 2020 · Databases

How to Use Elasticsearch Search Templates for Batch Terms Queries

This article explains why a terms query with an array fails inside an Elasticsearch search template, demonstrates the correct Mustache‑based template syntax, and provides step‑by‑step code examples for defining the index, inserting data, creating the template, and executing the query.

AggregationDSLElasticsearch
0 likes · 8 min read
How to Use Elasticsearch Search Templates for Batch Terms Queries
System Architect Go
System Architect Go
Nov 2, 2020 · Backend Development

Custom Scoring in Elasticsearch Using function_score

Elasticsearch calculates a relevance score for each document, but using the function_score query you can customize this scoring by combining the original query_score with a user-defined func_score through various functions such as weight, random_score, field_value_factor, decay_function, and script_score, allowing flexible ranking based on business needs.

BackendCustom ScoringElasticsearch
0 likes · 11 min read
Custom Scoring in Elasticsearch Using function_score
Top Architect
Top Architect
Oct 31, 2020 · Big Data

Building a Zhihu User Data Crawler and Large‑Scale Analysis with SpringBoot, SeimiCrawler, RabbitMQ, ElasticSearch, and Kibana

This article describes how to build a Java‑based crawler to collect millions of Zhihu user profiles, handle anti‑crawling measures with rotating user‑agents and a proxy pool, deduplicate data using a Bloom filter, import the results into ElasticSearch, and analyze the dataset with Kibana and ECharts visualizations.

ElasticsearchJavaKibana
0 likes · 15 min read
Building a Zhihu User Data Crawler and Large‑Scale Analysis with SpringBoot, SeimiCrawler, RabbitMQ, ElasticSearch, and Kibana
Java Architect Essentials
Java Architect Essentials
Oct 30, 2020 · Databases

Elasticsearch Essentials: Quick Start, Index Management, Mapping, and Advanced Operations

The article offers a thorough, step‑by‑step guide to Elasticsearch, explaining how to check cluster health, create and manage indices, define mappings and field types, use dynamic mapping, and perform maintenance tasks such as shrink, split, rollover, and cache management, all illustrated with concrete API examples.

ElasticsearchMappingTutorial
0 likes · 17 min read
Elasticsearch Essentials: Quick Start, Index Management, Mapping, and Advanced Operations
Tencent Cloud Developer
Tencent Cloud Developer
Oct 30, 2020 · Backend Development

Rebuilding QQ Mail Full-Text Search with Elasticsearch: Architecture, Implementation, and Optimization

To overcome aging hardware and code limitations, QQ Mail rebuilt its full‑text search using Tencent Cloud Elasticsearch, adding an esproxy layer, MQ‑driven updates, HTML‑to‑text extraction, protobuf‑JSON conversion, index sorting, two‑stage precise/fuzzy queries, and custom tokenizers, delivering scalable, low‑latency email search.

ElasticsearchProtobufSearch Optimization
0 likes · 17 min read
Rebuilding QQ Mail Full-Text Search with Elasticsearch: Architecture, Implementation, and Optimization
php Courses
php Courses
Oct 30, 2020 · Big Data

Introduction to Elasticsearch and Its Integration with Laravel

This article explains Elasticsearch's foundation on Lucene, compares its concepts to MySQL, describes inverted indexing, and provides a step‑by‑step guide for installing, configuring, and using the basemkhirat/elasticsearch Laravel plugin with code examples and tips for Chinese analysis.

BackendElasticsearchLaravel
0 likes · 4 min read
Introduction to Elasticsearch and Its Integration with Laravel
DataFunTalk
DataFunTalk
Oct 27, 2020 · Databases

Didi's Large‑Scale Elasticsearch Upgrade: Architecture, Migration Strategy, and Performance Gains

This article systematically details Didi's migration of over 30 Elasticsearch clusters, 3,500 nodes and 8 PB of data from version 2.3.3 to 6.6.1, covering background, problem analysis, multi‑version architecture redesign, capacity planning, tiered storage, FastIndex, query replay, upgrade pitfalls, and the resulting cost reduction and performance improvements.

CapacityPlanningElasticsearchPerformance
0 likes · 15 min read
Didi's Large‑Scale Elasticsearch Upgrade: Architecture, Migration Strategy, and Performance Gains
Python Crawling & Data Mining
Python Crawling & Data Mining
Oct 23, 2020 · Backend Development

Build a Flask‑Elasticsearch Search Engine: From Config to Deployment

This tutorial walks through building a Flask‑based search engine powered by Elasticsearch, covering configuration files, logging setup, blueprint routing with pagination, application initialization, and deployment options using Flask‑Script and Gunicorn, with complete code examples and a GitHub repository link.

ElasticsearchFlaskGunicorn
0 likes · 11 min read
Build a Flask‑Elasticsearch Search Engine: From Config to Deployment
DevOps Coach
DevOps Coach
Oct 19, 2020 · Backend Development

Understanding Elasticsearch Segment Merging: When and How to Use Force Merge

This article explains what Elasticsearch segments are, why they are immutable, how segment merging works, its impact on resources and search performance, and provides practical configuration tips such as using force_merge, refresh_interval adjustments, and thread count settings.

ElasticsearchForce MergeSegment Merging
0 likes · 9 min read
Understanding Elasticsearch Segment Merging: When and How to Use Force Merge
DevOps Coach
DevOps Coach
Oct 19, 2020 · Operations

Why Elasticsearch Query Latency Spikes Occur and How to Diagnose Them

This article examines the common causes of Elasticsearch query latency spikes—especially GC pauses, system cache misses, and I/O overhead—provides step‑by‑step methods to identify the root cause, and offers practical tuning recommendations to mitigate the issue.

ElasticsearchGCI/O
0 likes · 14 min read
Why Elasticsearch Query Latency Spikes Occur and How to Diagnose Them
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Oct 18, 2020 · Backend Development

Spring Boot 2.0.5 Integration with Elasticsearch: Installation, Configuration, Entity Mapping, CRUD, and Full‑Text Search

This guide demonstrates how to integrate Spring Boot 2.0.5 with Elasticsearch 5.6.11 by installing Elasticsearch and Kibana, adding the required Maven dependencies, configuring the client via application.yml or Java code, defining @Document‑annotated entities, implementing repository, service and controller layers for CRUD and full‑text search, handling Netty conflicts at startup, and verifying the setup with health, list and search endpoints.

Backend DevelopmentElasticsearchJava
0 likes · 8 min read
Spring Boot 2.0.5 Integration with Elasticsearch: Installation, Configuration, Entity Mapping, CRUD, and Full‑Text Search
MaGe Linux Operations
MaGe Linux Operations
Oct 12, 2020 · Databases

How to Diagnose and Fix Elasticsearch Cluster Health Issues

This guide explains how to monitor Elasticsearch cluster health, interpret green/yellow/red statuses, troubleshoot unassigned shards, adjust JVM and system settings, resolve common configuration errors, and use scripts and APIs to keep your ELK stack stable and performant.

ElasticsearchJVM TuningSplit-Brain Prevention
0 likes · 28 min read
How to Diagnose and Fix Elasticsearch Cluster Health Issues
Wukong Talks Architecture
Wukong Talks Architecture
Oct 9, 2020 · Big Data

Elasticsearch Fundamentals: Architecture, Indexing, Queries, Docker Setup, and Chinese Tokenization

This tutorial introduces Elasticsearch's core concepts, installation via Docker, index and document operations, query DSL, aggregations, and Chinese tokenization using the IK analyzer with custom dictionaries, providing step‑by‑step code examples for building a searchable log analysis stack.

Chinese tokenizationDockerElasticsearch
0 likes · 28 min read
Elasticsearch Fundamentals: Architecture, Indexing, Queries, Docker Setup, and Chinese Tokenization
Architecture Digest
Architecture Digest
Oct 1, 2020 · Big Data

Elasticsearch Overview: Architecture, Core Concepts, and Performance Optimization

This article provides a comprehensive introduction to Elasticsearch, covering data types, the role of Lucene, cluster architecture, node roles, discovery mechanisms, shard and replica management, mapping, installation, health monitoring, indexing workflow, storage internals, refresh and translog processes, segment merging, and practical performance and JVM tuning tips.

ElasticsearchPerformance OptimizationReplica
0 likes · 35 min read
Elasticsearch Overview: Architecture, Core Concepts, and Performance Optimization
dbaplus Community
dbaplus Community
Sep 28, 2020 · Databases

How ClickHouse Outperformed Elasticsearch for Ctrip’s Log Analytics: Speed, Cost & Ops

Facing growing log volumes and high Elasticsearch costs, Ctrip migrated its 200 TB daily logs to ClickHouse, achieving up to 38× faster queries, 60 % lower server resources, and simplified operations through columnar storage, sharding, and custom dashboards, while detailing deployment, tuning, and common pitfalls.

ClickHouseElasticsearchLog Analytics
0 likes · 14 min read
How ClickHouse Outperformed Elasticsearch for Ctrip’s Log Analytics: Speed, Cost & Ops
Tencent Cloud Developer
Tencent Cloud Developer
Sep 27, 2020 · Operations

Elasticsearch Cluster Capacity Planning, Index Configuration, and Performance Optimization

This guide outlines practical capacity‑planning, index‑design, and write‑performance tuning for Tencent Cloud Elasticsearch clusters, covering compute and storage sizing, optimal shard counts, rollover strategies, bulk API settings, health monitoring, and common troubleshooting steps to ensure stable, high‑throughput search services.

Cluster PlanningElasticsearchOperations
0 likes · 19 min read
Elasticsearch Cluster Capacity Planning, Index Configuration, and Performance Optimization
ITPUB
ITPUB
Sep 24, 2020 · Information Security

What Happens When Bing’s Mobile Apps Leak Over 6.5 TB of User Data?

A recent investigation revealed that an unsecured server containing more than 6.5 TB of user data from Bing’s iOS, iPadOS, and Android apps was exposed, allowing attackers to harvest nearly 100 million records and launch destructive “Meow” attacks that nearly wiped the Elasticsearch database.

BingElasticsearchMeow Attack
0 likes · 6 min read
What Happens When Bing’s Mobile Apps Leak Over 6.5 TB of User Data?
Java Architect Essentials
Java Architect Essentials
Sep 23, 2020 · Big Data

Evolution of JD.com Order Center Elasticsearch Cluster Architecture

The article details how JD.com's order center migrated its massive order query workload from MySQL to Elasticsearch, iteratively improving cluster isolation, node deployment, replica tuning, master‑slave redundancy, version upgrades, and data synchronization while addressing performance pitfalls such as deep pagination and FieldData usage.

Elasticsearchbig datacluster architecture
0 likes · 12 min read
Evolution of JD.com Order Center Elasticsearch Cluster Architecture
ITPUB
ITPUB
Sep 23, 2020 · Information Security

What the Bing Mobile Data Leak Means for Your Privacy

A massive breach exposed over 6.5 TB of Bing mobile app data—including search queries, location coordinates, device identifiers, and URLs—on an unsecured Elasticsearch server, putting iOS and Android users at risk of fraud, phishing, and physical threats, while highlighting common causes of such leaks.

BingElasticsearchUser Privacy
0 likes · 6 min read
What the Bing Mobile Data Leak Means for Your Privacy
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 18, 2020 · Big Data

Understanding the Elasticsearch Master Election Process

This article explains when Elasticsearch triggers a master election, describes each election stage—including active master and candidate selection, Bully algorithm comparison, and master node responsibilities—while providing code excerpts that illustrate the underlying implementation details.

ElasticsearchMaster Electionbig data
0 likes · 8 min read
Understanding the Elasticsearch Master Election Process
dbaplus Community
dbaplus Community
Sep 15, 2020 · Big Data

How Didi Doubled Elasticsearch Write Throughput and Cut Server Costs

Didi’s engineering team analyzed a severe write bottleneck in their 3000‑node Elasticsearch cluster, identified long‑tail latency caused by refresh, translog locks, write queues and GC, and applied routing‑aware bulk writes, JVM and Lucene tweaks, and data cleaning to more than double write throughput while slashing server costs.

Bulk APIDidiElasticsearch
0 likes · 17 min read
How Didi Doubled Elasticsearch Write Throughput and Cut Server Costs
Architecture Digest
Architecture Digest
Sep 14, 2020 · Databases

Understanding the Underlying Mechanics of Elasticsearch and Lucene

This article provides a comprehensive, top‑down and bottom‑up explanation of Elasticsearch’s internal architecture, covering clusters, nodes, shards, Lucene segments, inverted indexes, stored fields, document values, caching, merging, routing, scaling, and query processing, while addressing common performance questions.

CachingElasticsearchLucene
0 likes · 11 min read
Understanding the Underlying Mechanics of Elasticsearch and Lucene
Architect's Tech Stack
Architect's Tech Stack
Sep 13, 2020 · Big Data

Evolution and Optimization of JD Daojia Order Center Elasticsearch Cluster

This article details how JD Daojia's order center migrated from a simple MySQL‑backed system to a high‑throughput Elasticsearch cluster, describing each architectural phase, performance tuning measures, dual‑cluster real‑time backup, version upgrades, data synchronization strategies, and the key pitfalls encountered such as deep pagination and FieldData memory issues.

DocValuesElasticsearchPerformance Optimization
0 likes · 11 min read
Evolution and Optimization of JD Daojia Order Center Elasticsearch Cluster
Programmer DD
Programmer DD
Sep 13, 2020 · Backend Development

How JD.com Scaled Its Order System with Elasticsearch: Architecture Evolution

This article details how JD.com's order center migrated from MySQL‑only reads to a high‑throughput Elasticsearch cluster, describing each architectural phase—from the initial bare‑metal setup, through isolation, replica tuning, primary‑secondary adjustments, to the current real‑time dual‑cluster—while sharing synchronization strategies and performance pitfalls.

Elasticsearchcluster scalingdata synchronization
0 likes · 12 min read
How JD.com Scaled Its Order System with Elasticsearch: Architecture Evolution
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 11, 2020 · Big Data

Evolution of JD.com Order Center Elasticsearch Cluster Architecture

This article details how JD.com's order center migrated its Elasticsearch cluster from a simple, default‑configured setup to a highly available, multi‑replica, dual‑cluster architecture with version upgrades, data synchronization strategies, and performance optimizations to support billions of documents and hundreds of millions of daily queries.

ElasticsearchPerformance Optimizationbig data
0 likes · 12 min read
Evolution of JD.com Order Center Elasticsearch Cluster Architecture
Tencent Cloud Developer
Tencent Cloud Developer
Sep 8, 2020 · Backend Development

Implementing Autocomplete with MySQL, Redis, and Elasticsearch

The article explains autocomplete’s user‑friendly benefits and compares three backend approaches—simple MySQL LIKE queries, Redis sorted‑set range scans, and Elasticsearch’s completion suggester with FST indexing—highlighting their performance, scalability, and feature trade‑offs to help choose the best solution for a given dataset and latency requirement.

ElasticsearchMySQLautocomplete
0 likes · 8 min read
Implementing Autocomplete with MySQL, Redis, and Elasticsearch
Selected Java Interview Questions
Selected Java Interview Questions
Sep 3, 2020 · Operations

Evolution of JD.com Order Center Elasticsearch Cluster Architecture and Lessons Learned

This article details the progressive evolution of JD.com’s order center Elasticsearch cluster—from its initial default setup through isolation, replica optimization, master‑slave adjustments, and real‑time dual‑cluster backup—highlighting architectural decisions, scaling strategies, synchronization methods, and operational challenges encountered.

Elasticsearchcluster architecturedata synchronization
0 likes · 13 min read
Evolution of JD.com Order Center Elasticsearch Cluster Architecture and Lessons Learned
Architecture Digest
Architecture Digest
Sep 3, 2020 · Databases

Practical Elasticsearch Performance and Stability Tuning Guide

This article consolidates practical Elasticsearch tuning techniques—including configuration file adjustments, system‑level optimizations, and usage‑level settings—to improve cluster performance, stability, and resource efficiency for production environments.

Cluster ConfigurationElasticsearchbig data
0 likes · 15 min read
Practical Elasticsearch Performance and Stability Tuning Guide
Laravel Tech Community
Laravel Tech Community
Aug 31, 2020 · Big Data

Evolution of JD Daojia Order System Elasticsearch Cluster Architecture

This article details the step‑by‑step evolution of the JD Daojia order‑center Elasticsearch cluster—from an initial loosely configured deployment to a real‑time dual‑cluster architecture with replica tuning, master‑slave adjustments, data‑sync strategies, and lessons learned about pagination, fielddata, and doc values—highlighting how each phase improved query throughput, stability, and scalability for billions of documents.

ElasticsearchJD Daojiabig data
0 likes · 12 min read
Evolution of JD Daojia Order System Elasticsearch Cluster Architecture
Tencent Cloud Developer
Tencent Cloud Developer
Aug 27, 2020 · Big Data

Elasticsearch Overview: Architecture, Lucene Foundations, Application Scenarios, and Optimizations

Elasticsearch, built on Apache Lucene, provides a distributed, near‑real‑time search platform that scales to billions of documents across thousands of nodes, supporting use cases such as log analytics, time‑series monitoring, and product search, while Tencent’s CES adds advanced availability, performance, and cost‑optimizing features.

ElasticsearchLucenePerformance Optimization
0 likes · 17 min read
Elasticsearch Overview: Architecture, Lucene Foundations, Application Scenarios, and Optimizations
DevOps Coach
DevOps Coach
Aug 27, 2020 · Operations

How to Build a Secure 3‑Node Elasticsearch 7.9 Cluster with Vagrant

This guide walks through the new features of Elastic Stack 7.9 and shows step‑by‑step how to provision a three‑node Elasticsearch cluster on a local Mac using Vagrant, enabling security, TLS, API keys, and providing all necessary scripts, configuration files, and verification commands.

Elastic StackElasticsearchInstallation
0 likes · 12 min read
How to Build a Secure 3‑Node Elasticsearch 7.9 Cluster with Vagrant
Tencent Cloud Developer
Tencent Cloud Developer
Aug 26, 2020 · Databases

How to Become an Outstanding Data Craftsman: Insights from Database Experts

Three leading database experts from Tencent and cloud firms share how deep curiosity, hands‑on experimentation, persistent learning—combined with mastering distributed, cloud, and autonomous databases—forge an outstanding data craftsman, while advising newcomers to master performance, security, one DB per category, and start with open‑source tools.

Cloud DatabaseDBAElasticsearch
0 likes · 34 min read
How to Become an Outstanding Data Craftsman: Insights from Database Experts
Ops Development Stories
Ops Development Stories
Aug 25, 2020 · Operations

ESrally Guide: Install, Configure, and Benchmark Elasticsearch Performance

ESrally is the official Elasticsearch benchmarking tool; this guide walks through its installation prerequisites, step‑by‑step setup of Python, JDK, and Git, configuration of tracks, cars, pipelines, and challenges, and demonstrates real‑world performance comparisons across Elasticsearch versions and hardware platforms.

ESrallyElasticsearchOperations
0 likes · 16 min read
ESrally Guide: Install, Configure, and Benchmark Elasticsearch Performance
Efficient Ops
Efficient Ops
Aug 24, 2020 · Operations

How to Scale Elasticsearch for PB‑Level Game Logs: Real‑World Strategies & Lessons

This article walks through a mid‑size gaming company's journey of deploying, tuning, and scaling an Elasticsearch cluster for massive log volumes, covering hot‑cold node architecture, ILM policies, shard management, Logstash‑Kafka optimization, emergency expansions, and the promise of searchable snapshots to achieve petabyte‑scale storage with cost efficiency.

ElasticsearchILMLog Management
0 likes · 28 min read
How to Scale Elasticsearch for PB‑Level Game Logs: Real‑World Strategies & Lessons
DevOps Coach
DevOps Coach
Aug 24, 2020 · Operations

Deploy Elastic Stack Beats on Tencent Cloud for Scalable Monitoring

This guide walks you through creating a Tencent Cloud Elasticsearch cluster, configuring secure write‑only Beats roles, initializing Beats with setup commands, deploying Beats via a scripted workflow, and troubleshooting common issues to achieve reliable, scalable production monitoring.

BeatsDevOpsElastic Stack
0 likes · 9 min read
Deploy Elastic Stack Beats on Tencent Cloud for Scalable Monitoring
Ops Development Stories
Ops Development Stories
Aug 24, 2020 · Databases

How to Securely Backup Elasticsearch with Snapshots and S3 Repositories

This guide explains why Elasticsearch replicas cannot protect against catastrophic failures and shows step‑by‑step how to configure snapshot and restore using various repository types—including S3, HDFS, Azure, and Huawei OBS—install necessary plugins, set up JVM options, and automate snapshot policies via Kibana.

ElasticsearchKibanaOBS
0 likes · 9 min read
How to Securely Backup Elasticsearch with Snapshots and S3 Repositories
Programmer DD
Programmer DD
Aug 23, 2020 · Databases

What’s New in Elasticsearch 7.9.0? Key Security Fixes and Feature Updates

Elasticsearch 7.9.0 introduces critical security patches for field‑level leakage, updates script cache limits, refines field capabilities, improves snapshot restore throttling, expands thread‑pool write queue, deprecates dangling indices, and addresses known issues like mapping errors in machine‑learning indices.

Elasticsearchsearch engine
0 likes · 4 min read
What’s New in Elasticsearch 7.9.0? Key Security Fixes and Feature Updates
Programmer DD
Programmer DD
Aug 22, 2020 · Backend Development

Why Elasticsearch Can Be Slow and How to Supercharge Its Performance

This article examines common Elasticsearch interview questions, explains why initial searches can be slow, and provides practical strategies such as leveraging filesystem cache, data pre‑heating, cold‑hot index separation, minimal document design, and scroll or search_after APIs to dramatically improve search performance and pagination efficiency.

ElasticsearchFilesystem CachePerformance Optimization
0 likes · 13 min read
Why Elasticsearch Can Be Slow and How to Supercharge Its Performance
IT Architects Alliance
IT Architects Alliance
Aug 20, 2020 · Operations

Step-by-Step Guide to Deploying the ELK Stack with Docker Compose

This tutorial explains how to download Docker images for Elasticsearch, Kibana, and Logstash, create a Docker‑Compose configuration file, launch the ELK stack containers, verify their operation, customize Logstash to monitor log files, and finally use Kibana to search and analyze the collected logs.

DockerDocker ComposeELK
0 likes · 6 min read
Step-by-Step Guide to Deploying the ELK Stack with Docker Compose
Architects Research Society
Architects Research Society
Aug 17, 2020 · Databases

Interview with JanusGraph PMC Members on Graph Database Landscape, Neo4j Comparison, and Deployment Best Practices

In this interview, JanusGraph PMC members Florian Hockmann and Jason Plurad discuss the project's origins, compare JanusGraph with Neo4j, share advice for production deployments, outline future expectations for JanusGraph and TinkerPop, and provide practical tips for graph modeling and community contribution.

CassandraElasticsearchGremlin
0 likes · 16 min read
Interview with JanusGraph PMC Members on Graph Database Landscape, Neo4j Comparison, and Deployment Best Practices
DevOps Coach
DevOps Coach
Aug 13, 2020 · Databases

How to Benchmark Elasticsearch Clusters with Rally: A Step‑by‑Step Guide

This article explains why large‑scale Elasticsearch deployments need rigorous performance testing, compares available testing tools, walks through installing and configuring the official Rally benchmark suite, details hardware recommendations, shows how to run tests against multiple cloud providers, and teaches you how to interpret the resulting metrics to make informed cluster‑selection decisions.

Data AnalysisElasticsearchRally
0 likes · 16 min read
How to Benchmark Elasticsearch Clusters with Rally: A Step‑by‑Step Guide
Sohu Tech Products
Sohu Tech Products
Aug 12, 2020 · Big Data

Elasticsearch Basics: Concepts, Installation, and Search Operations

This article introduces Elasticsearch as a distributed open‑source search and analytics engine, explains its core concepts and architecture, compares it with relational databases, details installation steps, configuration, indexing, analyzers, query DSL, pagination, sorting, and provides practical examples for building search functionality.

AnalyzersElasticsearchInstallation
0 likes · 22 min read
Elasticsearch Basics: Concepts, Installation, and Search Operations
Didi Tech
Didi Tech
Aug 12, 2020 · Databases

Douyin ElasticSearch Write Performance Optimization: Doubling Throughput and Reducing Write Rejections

By consolidating each bulk request onto a single shard, tuning node write paths, and pruning unnecessary fields, Douyin’s ElasticSearch team eliminated long‑tail latency, more than doubled write throughput to over 1 million ops per second, slashed write rejections, and saved millions in infrastructure costs.

Bulk RequestElasticsearchLong Tail
0 likes · 18 min read
Douyin ElasticSearch Write Performance Optimization: Doubling Throughput and Reducing Write Rejections
Tencent Cloud Middleware
Tencent Cloud Middleware
Aug 12, 2020 · Big Data

How Serverless Functions Can Replace Traditional Kafka Data Pipelines for Lower Cost and Easier Scaling

This article explains how Tencent Cloud CKafka works, describes the challenges of traditional open‑source data‑flow solutions, and demonstrates a Serverless Function approach—complete with architecture diagrams and code examples—to achieve low‑cost, auto‑scaling Kafka‑to‑Elasticsearch pipelines.

CKafkaData PipelineElasticsearch
0 likes · 12 min read
How Serverless Functions Can Replace Traditional Kafka Data Pipelines for Lower Cost and Easier Scaling
macrozheng
macrozheng
Aug 10, 2020 · Backend Development

Mall Project Update: SpringBoot 2.3, Elasticsearch 7.6, Unified Swagger & Redis

This article outlines the latest upgrades to the open‑source Mall project, including migration to SpringBoot 2.3.0, support for Elasticsearch 7.6.2, enhanced ELK log collection, unified Swagger and Redis configurations, and updated deployment guides for Windows and Linux environments.

BackendElasticsearchOpenSource
0 likes · 10 min read
Mall Project Update: SpringBoot 2.3, Elasticsearch 7.6, Unified Swagger & Redis
Programmer DD
Programmer DD
Aug 8, 2020 · Artificial Intelligence

How Elasticsearch Handles Write, Read, and Search: Inside the Engine

This article explains Elasticsearch's internal mechanisms for indexing, querying, and retrieving data, covering the roles of coordinating nodes, primary and replica shards, the refresh and commit cycles, near‑real‑time search, and the underlying Lucene inverted index.

Data IngestionElasticsearchLucene
0 likes · 12 min read
How Elasticsearch Handles Write, Read, and Search: Inside the Engine
Efficient Ops
Efficient Ops
Aug 4, 2020 · Operations

Mastering Filebeat: How to Collect and Ship Container Logs to Kafka

This article introduces Filebeat as a lightweight log shipper, explains its core components and processing flow, and provides step‑by‑step configuration examples for gathering container logs and forwarding them to Kafka or Elasticsearch in cloud‑native environments.

ElasticsearchFilebeatGo
0 likes · 13 min read
Mastering Filebeat: How to Collect and Ship Container Logs to Kafka
Tencent Cloud Developer
Tencent Cloud Developer
Aug 4, 2020 · Cloud Computing

Tencent Cloud Elasticsearch Optimization Practices in Tencent Meeting: High Availability, Performance, and Cost-Effective Solutions

Tencent Meeting migrated its quality‑analysis system to Tencent Cloud Elasticsearch, tackling OOM failures, 3 M/s write spikes and scaling limits by adding multi‑AZ deployment, leaky‑bucket rate limiting, streaming aggregation checks, optimized merge and translog handling, plus hot‑warm storage, ILM, multi‑disk and off‑heap caching, cutting cluster size from 15 000 to under 300 nodes while maintaining high availability and performance.

ElasticsearchPerformance OptimizationTencent Cloud
0 likes · 23 min read
Tencent Cloud Elasticsearch Optimization Practices in Tencent Meeting: High Availability, Performance, and Cost-Effective Solutions
Python Crawling & Data Mining
Python Crawling & Data Mining
Aug 1, 2020 · Backend Development

Build a Flask‑Elasticsearch Search Engine: Config, Logging, Routing & Deployment Guide

This tutorial walks you through creating a Flask‑based search engine powered by Elasticsearch, covering configuration files, logging setup, blueprint routing with pagination, project startup using Flask‑Script, and production deployment with Gunicorn, complete with code examples and essential tips.

Backend DevelopmentElasticsearchFlask
0 likes · 11 min read
Build a Flask‑Elasticsearch Search Engine: Config, Logging, Routing & Deployment Guide
Laravel Tech Community
Laravel Tech Community
Jul 29, 2020 · Backend Development

Elasticsearch 7.8.1 Release Highlights and New Features

Elasticsearch 7.8.1 introduces a range of new capabilities such as literal SUM/MIN/MAX/AVG in SQL, enhanced authorization for apm_user, updated index creation logging, composable template renaming, additional machine‑learning aggregations, snapshot/restore optimizations, and an improved update API, all aimed at boosting search and analytics performance.

7.8.1Elasticsearchfeatures
0 likes · 2 min read
Elasticsearch 7.8.1 Release Highlights and New Features
21CTO
21CTO
Jul 29, 2020 · Backend Development

How JD.com Scaled Its Order Search with Elasticsearch: From Chaos to Real‑Time Dual Clusters

This article details how JD.com’s order center migrated from a MySQL‑only design to a high‑performance Elasticsearch cluster, evolving through isolation, replica tuning, master‑slave adjustments, and real‑time dual‑cluster architecture to achieve billions of documents, hundreds of millions of daily queries, and robust fault tolerance.

Backend EngineeringElasticsearchSearch Architecture
0 likes · 12 min read
How JD.com Scaled Its Order Search with Elasticsearch: From Chaos to Real‑Time Dual Clusters
Tencent Cloud Developer
Tencent Cloud Developer
Jul 29, 2020 · Big Data

Case Study: Optimizing Tencent Cloud Elasticsearch for High‑Volume Game Log Analytics

To handle a gaming company's million‑QPS log stream, the team built a hot‑cold Tencent Cloud Elasticsearch cluster with ILM‑driven tiering, scaled CPU/heap, reduced shard count via shrink and replica tweaks, tuned Logstash‑Kafka pipelines, and employed COS snapshots and searchable snapshots, achieving stable performance and lower cost.

ElasticsearchILMKafka
0 likes · 29 min read
Case Study: Optimizing Tencent Cloud Elasticsearch for High‑Volume Game Log Analytics
MaGe Linux Operations
MaGe Linux Operations
Jul 28, 2020 · Big Data

How Leading Chinese Companies Scale Elasticsearch for Billions of Orders

This article surveys how major Chinese tech firms such as JD.com, Ctrip, Didi, and 58.com deploy and evolve Elasticsearch clusters to handle massive order data, log analysis, real‑time monitoring, and security tasks, detailing architecture choices, shard strategies, multi‑cluster designs, and performance optimizations.

ElasticsearchSearch Architecturebig data
0 likes · 11 min read
How Leading Chinese Companies Scale Elasticsearch for Billions of Orders
转转QA
转转QA
Jul 23, 2020 · Operations

Building a Near Real‑Time Log Collection and Query System for Distributed Deployment

The article describes how a distributed deployment platform built a centralized Elasticsearch‑based log collection and query system to replace manual multi‑machine log inspection, detailing the background challenges, architecture, implementation steps, practical usage, and future improvements.

ElasticsearchKibanaLog Management
0 likes · 6 min read
Building a Near Real‑Time Log Collection and Query System for Distributed Deployment
Tencent Cloud Developer
Tencent Cloud Developer
Jul 22, 2020 · Backend Development

Practical Optimization of Elasticsearch Search Ranking

The article explains how to systematically improve Elasticsearch search relevance by fine‑tuning Query DSL with filters, phrase matching, and boosts, incorporating static scoring via function_score, adjusting BM25 similarity parameters, and using diagnostics like _explain to iteratively achieve higher ranking quality.

BM25BoostElasticsearch
0 likes · 17 min read
Practical Optimization of Elasticsearch Search Ranking
Tencent Cloud Developer
Tencent Cloud Developer
Jul 21, 2020 · Big Data

Scaling Tencent Meeting Video Stream Quality Analysis with Tencent Cloud Elasticsearch

Facing explosive growth and massive video‑stream quality data, Tencent Meeting migrated its custom Lucene‑based analysis engine to Tencent Cloud Elasticsearch, which delivered over 1 million writes per second, automatic sharding, reduced latency from hours to seconds, and sustained 99.99% availability, proving a high‑performance, scalable solution for large‑scale video conferencing.

Cloud ComputingElasticsearchPerformance Optimization
0 likes · 16 min read
Scaling Tencent Meeting Video Stream Quality Analysis with Tencent Cloud Elasticsearch
Tencent Cloud Developer
Tencent Cloud Developer
Jul 20, 2020 · Cloud Native

Tencent Eagle Eye Distributed Logging System Cloud Migration Practice

Tencent’s Eagle Eye distributed real‑time monitoring and log analysis platform was migrated to the cloud by rebuilding its LogSender and Kafka‑to‑ES components, switching to cloud CKafka and Elasticsearch, which boosted throughput fourfold, cut resource usage by about half, saved roughly 20 million RMB annually, and set the stage for further enhancements such as comprehensive monitoring and exactly‑once delivery.

Cloud-Native ArchitectureElasticsearchKafka
0 likes · 9 min read
Tencent Eagle Eye Distributed Logging System Cloud Migration Practice
DataFunTalk
DataFunTalk
Jul 16, 2020 · Big Data

Elasticsearch Practices and Platform Construction at 58.com

This article details 58.com’s extensive use of Elasticsearch for search, analytics, and log processing, covering cluster optimization challenges, typical issues like disk exhaustion and write slowdown, practical solutions, development standards, ELKB architecture, real‑time log and MySQL slow‑log applications, platform‑as‑a‑service construction, and future roadmap plans.

ElasticsearchLog AnalyticsPlatform Engineering
0 likes · 17 min read
Elasticsearch Practices and Platform Construction at 58.com
Jike Tech Team
Jike Tech Team
Jul 15, 2020 · Artificial Intelligence

How Embedding-Based Recall Boosted Interaction by 33% in a Live Feed

This article details how Jike's recommendation team upgraded from Spark to TensorFlow, introduced a twin‑tower embedding model for recall, deployed it with TensorFlow Serving and Elasticsearch, and achieved a 33.75% lift in user interaction on the dynamic square.

ElasticsearchTensorFlow Servingdeep learning
0 likes · 9 min read
How Embedding-Based Recall Boosted Interaction by 33% in a Live Feed
Swan Home Tech Team
Swan Home Tech Team
Jul 13, 2020 · Backend Development

Design and Evolution of the DaJia App Search System

This article explains the motivations, requirements, and technical design of the DaJia app's search system, compares relational databases with Lucene‑based solutions, describes the inverted index mechanism, outlines common search workflows, and details the system's three iterative development phases and future improvement plans.

BackendElasticsearchInformation Retrieval
0 likes · 12 min read
Design and Evolution of the DaJia App Search System