BestHub
Sign in
Tagged articles

log management

191 articles · Page 2 of 2
DataFunSummit
DataFunSummit
Jul 17, 2022 · Big Data

Elasticsearch and Big Data: Architecture, Use Cases, and Advantages

This article explains what Elasticsearch is, how it solves database acceleration, log observability, and data analysis problems, details its core components and underlying engine features, compares its strengths and weaknesses, and presents classic application scenarios and a real‑world case study integrating Elasticsearch with Flink for large‑scale log analytics.

Big DataCase StudyElasticsearch
0 likes · 13 min read
Elasticsearch and Big Data: Architecture, Use Cases, and Advantages
dbaplus Community
dbaplus Community
May 24, 2022 · Big Data

How Vipshop Replaced ELK with ClickHouse for a Scalable, Low‑Cost Log System

Vipshop’s Dragonfly log platform evolved from a costly 260‑node Elasticsearch cluster to a ClickHouse‑based architecture that uses a unified JSON format, vfilebeat ingestion, Flink parsing, and MergeTree storage to achieve high‑throughput writes, fast vectorized queries, flexible TTL management, and dramatically lower operational expenses.

ClickHouseEFKFlink
0 likes · 20 min read
How Vipshop Replaced ELK with ClickHouse for a Scalable, Low‑Cost Log System
Top Architect
Top Architect
May 12, 2022 · Operations

ELK Stack: Common Deployment Architectures, Configuration Tips, and Troubleshooting

This article introduces the ELK stack as a popular centralized logging solution, explains three typical deployment architectures—including Logstash‑based, Filebeat‑based, and cache‑queue‑enhanced setups—details their advantages and drawbacks, and provides practical configuration examples and troubleshooting techniques for multiline logs, timestamp handling, and module filtering.

ELKElasticsearchKibana
0 likes · 9 min read
ELK Stack: Common Deployment Architectures, Configuration Tips, and Troubleshooting
Architect
Architect
May 8, 2022 · Operations

ELK Stack Common Deployment Architectures and Practical Solutions

This article introduces the ELK stack components, compares three typical deployment architectures—Logstash as collector, Filebeat as collector, and a cache‑queue‑enhanced design—then discusses common logging issues such as multiline merging, timestamp handling, and module filtering, providing concrete configuration examples and solutions.

ELKKibanaLogstash
0 likes · 10 min read
ELK Stack Common Deployment Architectures and Practical Solutions
Volcano Engine Developer Services
Volcano Engine Developer Services
Apr 26, 2022 · Operations

How Volcano Engine’s TLS Transforms Log Management for Kubernetes at Scale

This article explains the challenges of traditional open‑source log collection in cloud‑native environments, describes Volcano Engine’s unified TLS architecture, its centralized configuration, CRD‑based deployment, and showcases real‑world case studies that demonstrate improved availability, efficiency, and scalability.

Cloud NativeKubernetesObservability
0 likes · 15 min read
How Volcano Engine’s TLS Transforms Log Management for Kubernetes at Scale
MaGe Linux Operations
MaGe Linux Operations
Apr 7, 2022 · Operations

Master ELK Stack: From Basics to Full‑Scale Log Management

This article introduces the ELK stack components, explains why centralized logging is essential, outlines core log‑system features, compares three ELK architectures, provides step‑by‑step installation and configuration for Filebeat, Logstash, Elasticsearch and Kibana, and demonstrates a complete pipeline using Kafka with code examples and diagrams.

ELKElasticsearchKafka
0 likes · 25 min read
Master ELK Stack: From Basics to Full‑Scale Log Management
Code Ape Tech Column
Code Ape Tech Column
Apr 1, 2022 · Operations

ELK Stack Common Deployment Architectures and Practical Solutions

This article introduces the ELK stack components, compares three typical deployment architectures—including Logstash‑based, Filebeat‑based, and Kafka‑enhanced setups—and provides detailed configuration examples and solutions for multiline log merging, timestamp handling, and module‑specific filtering.

ELKElasticsearchKibana
0 likes · 10 min read
ELK Stack Common Deployment Architectures and Practical Solutions
Architecture Digest
Architecture Digest
Jan 10, 2022 · Operations

Comprehensive Guide to Deploying Filebeat and Graylog for Centralized Log Collection

This article explains how to use Filebeat and Graylog together for centralized log collection, covering Filebeat’s role, configuration files, input modules, Graylog’s architecture, pipeline rules, and step‑by‑step deployment using Docker and docker‑compose, providing practical commands and examples for operational environments.

DockerElasticsearchGraylog
0 likes · 14 min read
Comprehensive Guide to Deploying Filebeat and Graylog for Centralized Log Collection
Efficient Ops
Efficient Ops
Dec 13, 2021 · Cloud Native

How I Built an Enterprise‑Grade Kubernetes Cluster Architecture from Scratch

This article recounts a year‑long journey of designing, implementing, and operating a multi‑environment Kubernetes architecture—including containerized workloads, unified logging, CI/CD pipelines, service governance with Istio, and private deployments—while sharing practical lessons and best‑practice recommendations for cloud‑native teams.

CI/CDCloud NativeService Governance
0 likes · 9 min read
How I Built an Enterprise‑Grade Kubernetes Cluster Architecture from Scratch
Programmer DD
Programmer DD
Dec 8, 2021 · Operations

Master Log Collection with Filebeat and Graylog: A Step‑by‑Step Guide

This article explains why centralized log collection is essential for multi‑environment services, introduces Graylog’s architecture, details Filebeat’s role and workflow, provides configuration examples, and walks through Docker‑based deployment of both Filebeat and Graylog for robust log management.

DockerElasticsearchGraylog
0 likes · 14 min read
Master Log Collection with Filebeat and Graylog: A Step‑by‑Step Guide
Dada Group Technology
Dada Group Technology
Sep 10, 2021 · Operations

Design and Implementation of JD Daojia Log System Based on Loki

This document details the motivation, architecture, components, query language, and deployment of a Loki‑based log collection and analysis platform for JD Daojia, comparing it with ELK, describing ingestion, real‑time and historical log handling, technical challenges, configuration examples, and future scaling plans.

CassandraGrafanaObservability
0 likes · 15 min read
Design and Implementation of JD Daojia Log System Based on Loki
Liangxu Linux
Liangxu Linux
Aug 15, 2021 · Operations

Master Logrotate: Automate Linux Log Rotation and Ditch Custom Scripts

This guide explains how to use the built‑in Logrotate tool on Linux to configure automatic log rotation, customize rotation policies for services like Nginx, PHP and syslog, troubleshoot common issues, and replace ad‑hoc scripts with reliable, cron‑driven solutions.

Linuxlog managementlogrotate
0 likes · 23 min read
Master Logrotate: Automate Linux Log Rotation and Ditch Custom Scripts
Java Architecture Diary
Java Architecture Diary
Aug 11, 2021 · Operations

Unlock Loki v2.3.0: Custom Retention, Deletion & Recording Rules Explained

Version 2.3.0 of Loki introduces enhanced features such as per‑tenant custom retention policies, time‑range log deletion via the Compactor API, Prometheus‑style recording rules, a new pattern parser for LogQL, ingestion sharding for faster queries, and advanced IP‑matching syntax, all aimed at improving storage efficiency, compliance, and observability.

LogQLObservabilitySharding
0 likes · 9 min read
Unlock Loki v2.3.0: Custom Retention, Deletion & Recording Rules Explained
Top Architect
Top Architect
Aug 10, 2021 · Operations

Building and Using an ELK Real‑Time Log Analysis Platform

This tutorial explains how to set up a real‑time ELK log analysis platform, covering the architecture of Elasticsearch, Logstash and Kibana, detailed installation commands, configuration for Spring Boot and Nginx logs, and how to run the stack continuously with Supervisor.

ELKElasticsearchKibana
0 likes · 18 min read
Building and Using an ELK Real‑Time Log Analysis Platform
Open Source Linux
Open Source Linux
May 6, 2021 · Cloud Native

Why Loki Beats ELK for Cloud‑Native Log Management

This article explains how Loki, a lightweight, Prometheus‑compatible logging system, addresses the high resource cost, complexity, and operational overhead of traditional ELK/EFK stacks by using label‑based indexing, efficient compression, and scalable architecture for container‑cloud environments.

Cloud NativeELK alternativeObservability
0 likes · 7 min read
Why Loki Beats ELK for Cloud‑Native Log Management
IT Architects Alliance
IT Architects Alliance
May 1, 2021 · Big Data

Comprehensive Guide to ELK Stack (Elasticsearch, Logstash, Kibana) Installation, Configuration, and Architecture

This article provides a detailed overview of the ELK stack—including Elasticsearch, Logstash, Kibana, and Beats—explaining its components, why to use it for centralized log management, various deployment architectures, system tuning, security setup, and step‑by‑step installation and configuration commands for a production‑grade environment.

Big DataELKElasticsearch
0 likes · 22 min read
Comprehensive Guide to ELK Stack (Elasticsearch, Logstash, Kibana) Installation, Configuration, and Architecture
Programmer DD
Programmer DD
Apr 18, 2021 · Information Security

Why Your Enterprise Needs a Log Auditing System: Core Features & Deployment

This article explains what log auditing is, why a unified audit platform is essential for compliance and security operations, outlines its core goals, main functions such as unified collection, correlation analysis, real‑time alerts, forensic reporting, and describes typical deployment architectures and modules.

Log AuditingSIEMSecurity Monitoring
0 likes · 8 min read
Why Your Enterprise Needs a Log Auditing System: Core Features & Deployment
Xianyu Technology
Xianyu Technology
Mar 9, 2021 · Mobile Development

Xianyu Android Log Governance and Performance Monitoring Solution

The Xianyu Android Log Governance solution unifies local console logs, TLog and online logs, adds comprehensive logcat capture and AUS‑OSS retrieval, replaces BlockCanary with low‑overhead frame‑callback lag/ANR detection, and provides dashboards and a batch‑query platform, cutting technical‑opinion ratio from 10.5 % to 4.7 % and boosting upload success.

ANR detectionAndroidlog management
0 likes · 13 min read
Xianyu Android Log Governance and Performance Monitoring Solution
Didi Tech
Didi Tech
Feb 25, 2021 · Industry Insights

Why DiDi’s Obsuite Is Redefining Hybrid‑Cloud Observability

Obsuite, DiDi’s open‑source observability suite, tackles hybrid‑cloud monitoring challenges by combining metrics, logs, and traces, while the article analyzes market trends, private‑cloud demand, and the product’s architecture, open‑source components, and the OCE certification program for enterprise users.

Hybrid CloudIndustry TrendsObservability
0 likes · 6 min read
Why DiDi’s Obsuite Is Redefining Hybrid‑Cloud Observability
Architecture Digest
Architecture Digest
Feb 15, 2021 · Operations

ELK Stack Overview, Architecture, Installation and Configuration Guide (Version 7.7.0)

This article provides a comprehensive introduction to the ELK stack—Elasticsearch, Logstash, Kibana, and Filebeat—including its components, why it’s used for centralized log management, detailed architecture diagrams, step‑by‑step installation commands, configuration examples, and a practical Kafka‑based data pipeline demonstration.

Big DataELKElasticsearch
0 likes · 22 min read
ELK Stack Overview, Architecture, Installation and Configuration Guide (Version 7.7.0)
dbaplus Community
dbaplus Community
Feb 2, 2021 · Backend Development

How to Diagnose Disk‑Space Exhaustion During a Traffic Surge and Build a Dynamic Log‑Level Degradation Tool

During a high‑traffic promotion a service ran out of disk space because massive log files weren't cleaned, the investigation revealed a lingering SLS process holding deleted files, and the article walks through the root‑cause analysis, the kill‑process fix, and a Spring‑Boot starter that dynamically degrades log levels to prevent recurrence.

Spring Bootdisk spacedynamic logging
0 likes · 20 min read
How to Diagnose Disk‑Space Exhaustion During a Traffic Surge and Build a Dynamic Log‑Level Degradation Tool
Youzan Coder
Youzan Coder
Dec 30, 2020 · Operations

ERROR Log Governance and Monitoring Alerting Practice at Youzan

Youzan’s log‑governance guide uses a car‑dashboard analogy to show why precise ERROR logs and sensible alerts matter, defines INFO/WARN/ERROR levels, sets daily reduction targets, leverages top‑error analysis and water‑level monitoring, and ultimately cut daily ERROR entries from thousands to about one hundred while catching issues before incidents.

AlertingError handlingOperations
0 likes · 9 min read
ERROR Log Governance and Monitoring Alerting Practice at Youzan
Laravel Tech Community
Laravel Tech Community
Nov 25, 2020 · Operations

Comprehensive Guide to Logrotate: Configuration, Usage, and Custom Scripts for Linux Log Management

This article provides an in‑depth tutorial on using the Linux logrotate utility, covering default configuration files, command‑line options, example configurations for services like Nginx, PHP‑FPM, Tomcat, custom Bash and Python rotation scripts, troubleshooting tips, and how to adjust cron scheduling for automated log rotation.

LinuxShell Scriptingcron
0 likes · 21 min read
Comprehensive Guide to Logrotate: Configuration, Usage, and Custom Scripts for Linux Log Management
Programmer DD
Programmer DD
Oct 26, 2020 · Operations

Master Logrotate: Complete Guide to Linux Log Rotation and Custom Scripts

This article provides a comprehensive walkthrough of Logrotate on Linux, covering its purpose, default configuration files, key directives, practical examples for Nginx, PHP, Tomcat, and system logs, as well as custom shell and Python scripts for advanced log rotation and troubleshooting.

Shell scriptcronlog management
0 likes · 22 min read
Master Logrotate: Complete Guide to Linux Log Rotation and Custom Scripts
ITPUB
ITPUB
Oct 10, 2020 · Operations

Master Logrotate: Complete Guide to Linux Log Rotation, Configuration, and Custom Scripts

This article provides a thorough walkthrough of Linux logrotate, covering its default configuration files, essential command‑line options, key parameters, practical examples for services like nginx, php, and tomcat, custom rotation scripts, cron scheduling, and troubleshooting tips for reliable log management.

LinuxNGINXShell script
0 likes · 24 min read
Master Logrotate: Complete Guide to Linux Log Rotation, Configuration, and Custom Scripts
Liangxu Linux
Liangxu Linux
Oct 6, 2020 · Operations

Master Linux Log Rotation with Logrotate: Configs, Scripts, and Common Fixes

This guide explains how to use Logrotate on Linux for automatic log file rotation, covering default configuration files, key options, example setups for Nginx, PHP, Tomcat, system logs, custom shell and Python scripts, cron scheduling, and troubleshooting techniques to ensure logs are rotated reliably.

LinuxNGINXcron
0 likes · 23 min read
Master Linux Log Rotation with Logrotate: Configs, Scripts, and Common Fixes
Efficient Ops
Efficient Ops
Aug 24, 2020 · Operations

How to Scale Elasticsearch for PB‑Level Game Logs: Real‑World Strategies & Lessons

This article walks through a mid‑size gaming company's journey of deploying, tuning, and scaling an Elasticsearch cluster for massive log volumes, covering hot‑cold node architecture, ILM policies, shard management, Logstash‑Kafka optimization, emergency expansions, and the promise of searchable snapshots to achieve petabyte‑scale storage with cost efficiency.

Big DataElasticsearchILM
0 likes · 28 min read
How to Scale Elasticsearch for PB‑Level Game Logs: Real‑World Strategies & Lessons
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 24, 2020 · Big Data

Key Concepts and Internal Mechanisms of Apache Kafka

This article provides an in‑depth overview of Apache Kafka’s internal topics, preferred replicas, partition allocation mechanisms, log directory structure, index files, offset and timestamp lookup, log retention and compaction policies, storage architecture, delayed operations, controller role, consumer rebalance process, and producer idempotence.

Consumer RebalanceIdempotenceKafka
0 likes · 18 min read
Key Concepts and Internal Mechanisms of Apache Kafka
转转QA
转转QA
Jul 23, 2020 · Operations

Building a Near Real‑Time Log Collection and Query System for Distributed Deployment

The article describes how a distributed deployment platform built a centralized Elasticsearch‑based log collection and query system to replace manual multi‑machine log inspection, detailing the background challenges, architecture, implementation steps, practical usage, and future improvements.

ElasticsearchKibanadistributed systems
0 likes · 6 min read
Building a Near Real‑Time Log Collection and Query System for Distributed Deployment
Top Architect
Top Architect
Jul 11, 2020 · Operations

7 Key Performance Metrics for Application Monitoring and Their Recommended Tools

The article outlines seven essential performance metrics—response time and throughput, average load, error rate, GC pause time, business indicators, uptime, and log size—explaining their significance for application health and recommending popular monitoring tools for each metric.

Garbage CollectionResponse TimeThroughput
0 likes · 6 min read
7 Key Performance Metrics for Application Monitoring and Their Recommended Tools
MaGe Linux Operations
MaGe Linux Operations
May 21, 2020 · Operations

Master Logrotate: Stop Disk‑Full Crises with Automated Log Rotation

This step‑by‑step guide shows how to diagnose exploding log files, configure logrotate with size‑based and time‑based rules, use cron jobs for frequent rotations, and apply options like copytruncate, delaycompress, and dateext to keep production systems healthy and storage under control.

cronlog managementlogrotate
0 likes · 29 min read
Master Logrotate: Stop Disk‑Full Crises with Automated Log Rotation
dbaplus Community
dbaplus Community
Apr 12, 2020 · Databases

Why and How to Migrate from MongoDB to Elasticsearch: A Practical Guide

This article explains the motivations for moving a high‑volume operation‑log system from MongoDB to Elasticsearch, outlines the existing architecture, details capacity planning, index design, and a step‑by‑step migration process using Kafka, DataX, and Spring Boot, and shares the performance gains and lessons learned.

Data MigrationDataXDatabase Architecture
0 likes · 14 min read
Why and How to Migrate from MongoDB to Elasticsearch: A Practical Guide
Youzan Coder
Youzan Coder
Apr 3, 2020 · Mobile Development

How ZanLogger Revolutionizes Mobile App Debugging with High‑Performance Log Management

The article explains the design and implementation of Youzan’s ZanLogger, a mobile logging platform that streamlines issue diagnosis by providing secure, efficient log collection, retrieval, and analysis features—including active and passive reporting, mmap‑based I/O reduction, dynamic symmetric key encryption, and multi‑dimensional storage—while detailing performance benchmarks and future enhancements.

AndroidPerformancelog management
0 likes · 12 min read
How ZanLogger Revolutionizes Mobile App Debugging with High‑Performance Log Management
37 Interactive Technology Team
37 Interactive Technology Team
Sep 27, 2019 · Operations

Centralized Management of Cron Jobs: Challenges and Solutions

The article outlines how a company built a centralized cron‑job platform—using Python’s crontab library, SaltStack deployment, ELK log aggregation, and automated email alerts—to integrate existing tasks, provide reliable CRUD operations, enable fast log querying, and detect failures, cutting operational overhead while managing thousands of scheduled jobs across multiple servers.

OperationsPythoncron
0 likes · 8 min read
Centralized Management of Cron Jobs: Challenges and Solutions
Java Captain
Java Captain
Aug 3, 2019 · Operations

Practical Guide to Viewing Logs, Processes, Ports, and System Status on Linux

This article provides a comprehensive, step‑by‑step tutorial on using Linux command‑line tools such as cat, tail, vim, grep, sed, ps, netstat, lsof, and free to efficiently view large log files, locate specific entries, monitor processes and ports, and assess overall system health.

LinuxOperationslog management
0 likes · 8 min read
Practical Guide to Viewing Logs, Processes, Ports, and System Status on Linux
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 28, 2019 · Operations

Comprehensive Guide to Building an ELK Log Management Platform with Kafka and Filebeat

This article provides a detailed tutorial on designing, deploying, and operating an ELK log management platform—including Elasticsearch, Logstash, Kibana, Kafka, and Filebeat—covering architecture options, configuration files, command‑line operations, cluster setup, and best‑practice recommendations for scalable, real‑time log collection and analysis.

ELKElasticsearchKafka
0 likes · 22 min read
Comprehensive Guide to Building an ELK Log Management Platform with Kafka and Filebeat
Architecture Digest
Architecture Digest
Jun 19, 2019 · Big Data

Design and Optimization of Large‑Scale Log Systems for High‑Volume Data

This article examines the challenges of handling massive log data in large‑scale e‑commerce platforms, outlines a baseline ELK‑based architecture, discusses real‑time versus near‑real‑time requirements, and presents four optimization strategies—including basic tuning, platform scaling, data partitioning, and system degradation—to improve performance, resource utilization, and reliability.

ELKSystem Optimizationlog management
0 likes · 17 min read
Design and Optimization of Large‑Scale Log Systems for High‑Volume Data
Youzan Coder
Youzan Coder
Apr 12, 2019 · Industry Insights

How Youzan Scaled Its Log Platform to Handle Billions of Daily Logs

This article details Youzan's evolution from a simple Flume‑based log collector to a multi‑tenant, Kafka‑buffered, Spark‑processed, HBase‑backed logging architecture that now handles hundreds of billions of log entries per day, highlighting challenges, design decisions, and future improvements.

ElasticsearchHBaseKafka
0 likes · 10 min read
How Youzan Scaled Its Log Platform to Handle Billions of Daily Logs
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jan 29, 2019 · Operations

How to Optimize Large-Scale Log Systems for Real-Time Monitoring and Scalability

This article examines the design, deployment, and optimization of massive log systems, comparing architectures, discussing real‑time versus near‑real‑time requirements, and presenting practical improvements such as memory, CPU, network tuning, data partitioning, storage reduction, and component upgrades using ELK, Kafka, Fluentd, and HBase.

Big DataELKFluentd
0 likes · 18 min read
How to Optimize Large-Scale Log Systems for Real-Time Monitoring and Scalability
Efficient Ops
Efficient Ops
Nov 22, 2018 · Artificial Intelligence

How AI Transforms Log Management: Building an Intelligent Log Center for AIOps

This article explores how AI-driven AIOps can turn massive operational log data into actionable insights, detailing the five‑level AI capability model, real‑world implementation scenarios, and industry case studies that demonstrate the value of an intelligent log center.

AI OperationsAIOpsIT Operations
0 likes · 10 min read
How AI Transforms Log Management: Building an Intelligent Log Center for AIOps
Liangxu Linux
Liangxu Linux
Oct 17, 2018 · Operations

Automatically Split and Rotate Large Linux Log Files

This guide shows how to automatically split growing Linux log files into 60 MB chunks with date‑based naming, using the `date` and `split` commands in a Bash script, and then safely truncate the original log to free disk space.

log managementsplit
0 likes · 5 min read
Automatically Split and Rotate Large Linux Log Files
HomeTech
HomeTech
Sep 25, 2018 · Operations

Design and Implementation of an Integrated Log Collection, Analysis, and Monitoring System

This article describes how a rapidly growing technical team built a unified log system that consolidates program, web access, and slow logs, introduces host‑agent and process‑agent collection, leverages Kafka, Elasticsearch, and Storm for high‑throughput processing, and provides monitoring, alerting, and reporting features to improve reliability and operational efficiency.

Big DataElasticsearchlog management
0 likes · 20 min read
Design and Implementation of an Integrated Log Collection, Analysis, and Monitoring System
Efficient Ops
Efficient Ops
Sep 19, 2018 · Cloud Native

Kubernetes Log Management: Challenges, Logtail Solution & Architecture

Amid the rise of serverless Kubernetes, growing pod volumes, and real-time log demands, this article examines emerging log-handling challenges, evaluates traditional collection methods, and presents a comprehensive “Logtail + Log Service + Ecosystem” architecture that delivers high-throughput, reliable, and scalable logging for cloud-native environments.

Cloud NativeKubernetesObservability
0 likes · 22 min read
Kubernetes Log Management: Challenges, Logtail Solution & Architecture
Architecture Digest
Architecture Digest
Jun 18, 2018 · Operations

Design and Optimization of Large‑Scale Log Systems

This article examines the challenges of handling massive log data in high‑traffic e‑commerce platforms and presents a comprehensive architecture, optimization strategies, and practical implementations—including Rsyslog, Kafka, Fluentd, and the ELK stack—to improve scalability, performance, and reliability of log management systems.

Big DataELKFluentd
0 likes · 17 min read
Design and Optimization of Large‑Scale Log Systems
Efficient Ops
Efficient Ops
Apr 8, 2018 · Operations

Why ELK Is the Ultimate Solution for Log Management and Monitoring

This article introduces the ELK stack—Elasticsearch, Logstash, and Kibana—explaining its core components, architecture, comparison with databases and grep, typical use cases across security, networking, and application monitoring, deployment considerations, challenges, SaaS prospects, and recommended learning resources.

ELKElasticsearchOperations
0 likes · 10 min read
Why ELK Is the Ultimate Solution for Log Management and Monitoring
Efficient Ops
Efficient Ops
Oct 18, 2017 · Operations

How Bilibili Scaled Its Log System to 10TB Daily with Elastic Stack

This article details Bilibili's Billions log platform—from its fragmented origins and design goals to the elastic‑stack‑based architecture, shard management, log sampling, custom Go splitters, and monitoring enhancements—highlighting the challenges faced and the roadmap for future improvements.

Big DataElastic StackOperations
0 likes · 17 min read
How Bilibili Scaled Its Log System to 10TB Daily with Elastic Stack
Hujiang Technology
Hujiang Technology
Oct 17, 2017 · Operations

Design and Implementation of a Distributed Real-Time Log Collection and Analysis System Using the ELK/EFK Stack

This article describes the background, requirements, architecture choices, performance testing, and lessons learned from building a large‑scale, distributed log collection and analysis platform at Hujiang using Elasticsearch, Logstash, Kibana, Filebeat, and Kafka to handle billions of log entries daily.

ELKKafkadistributed logging
0 likes · 12 min read
Design and Implementation of a Distributed Real-Time Log Collection and Analysis System Using the ELK/EFK Stack
ITPUB
ITPUB
Jun 15, 2017 · Operations

Mastering Logrotate: Automate Linux Log Rotation and Retention

This guide explains why logrotate is essential for Linux system security, details its default configuration files, shows how the cron‑based rotation works, lists key options, provides practical Nginx and custom rotation examples, and highlights the critical copytruncate option for seamless log handling.

Automationlog managementlogrotate
0 likes · 12 min read
Mastering Logrotate: Automate Linux Log Rotation and Retention
dbaplus Community
dbaplus Community
Jun 6, 2017 · Operations

Why Solid Foundational Data Is the Key to Stable Operations Management

This article explains how building complete, accurate, and real‑time foundational data—covering CMDB, logs, databases, and knowledge bases—forms the essential base for reliable IT operations, detailing the three core attributes, practical collection methods, and the structure of each data module.

CMDBData AccuracyIT Asset Management
0 likes · 15 min read
Why Solid Foundational Data Is the Key to Stable Operations Management
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
May 11, 2017 · Operations

How Heka + ELK Supercharges Mobile Game Log Management

Discover how integrating Heka with the ELK stack—Elasticsearch, Logstash, and Kibana—enables mobile game developers to efficiently collect, store, analyze, and visualize logs, offering rapid troubleshooting, automated alerts, and scalable AIOps capabilities that boost operational responsiveness and game performance.

AIOpsELKHeka
0 likes · 9 min read
How Heka + ELK Supercharges Mobile Game Log Management
Efficient Ops
Efficient Ops
May 1, 2017 · Backend Development

Why Ignoring Proper Logging Will Crash Your App—and How to Fix It

This article explains why developers must adopt mature logging frameworks, use log levels, split logs by time or size, choose appropriate storage backends, and consider unified platforms and real‑time analysis tools like ELK to ensure reliable debugging and monitoring in production.

ELKLoggingbackend
0 likes · 8 min read
Why Ignoring Proper Logging Will Crash Your App—and How to Fix It
MaGe Linux Operations
MaGe Linux Operations
Mar 26, 2017 · Operations

Deploy an ELK Stack Quickly with Docker Compose

This guide explains how to set up the ELK (Elasticsearch, Logstash, Kibana) logging platform using Docker Compose, covering prerequisites, environment configuration, single‑host and multi‑host deployments, and verification steps to get a functional ELK stack in minutes.

DockerELKElasticsearch
0 likes · 6 min read
Deploy an ELK Stack Quickly with Docker Compose
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Feb 23, 2017 · Operations

How to Build a Centralized Log System with the ELK Stack: Step‑by‑Step Installation Guide

This article explains the concept of centralized logging, introduces the ELK stack (Elasticsearch, Logstash, Kibana), compares it with other log solutions, and provides a detailed, cross‑platform installation and configuration guide for Ubuntu 14.04 and CentOS 7.1, including SSL setup and Nginx reverse‑proxy.

ELKElasticsearchInstallation
0 likes · 18 min read
How to Build a Centralized Log System with the ELK Stack: Step‑by‑Step Installation Guide
Meituan Technology Team
Meituan Technology Team
Feb 17, 2017 · Backend Development

How Dynamic Log Level Adjustment Boosts Reliability in High‑Throughput Delivery Services

The article explains the design, implementation, and operational benefits of a middleware component that dynamically adjusts log levels in Java services, helping large‑scale delivery platforms balance comprehensive debugging information with system performance and avoid catastrophic failures during peak traffic.

Backend PerformanceMiddlewareSLF4J
0 likes · 13 min read
How Dynamic Log Level Adjustment Boosts Reliability in High‑Throughput Delivery Services
dbaplus Community
dbaplus Community
Jul 18, 2016 · Operations

Integrated Monitoring for Securities: Solving Challenges, Defining Standards, and Measuring Success

The article gathers expert insights on building an integrated monitoring system for the securities industry, covering common pitfalls, the need for standardization, architectural design principles, KPI definitions, trend analysis techniques, and practical tool recommendations for effective operations.

KPIStandardizationintegrated monitoring
0 likes · 19 min read
Integrated Monitoring for Securities: Solving Challenges, Defining Standards, and Measuring Success
Architects' Tech Alliance
Architects' Tech Alliance
Apr 17, 2016 · Big Data

An Overview of Splunk as an IT Search Engine for Big Data Analysis

This article introduces Splunk as a specialized IT search engine for big data, detailing its data collection, indexing, analysis, reporting, security integration, scalability, and practical use cases such as operational troubleshooting, compliance auditing, and application performance monitoring.

Splunkdata analysislog management
0 likes · 7 min read
An Overview of Splunk as an IT Search Engine for Big Data Analysis
Efficient Ops
Efficient Ops
Feb 1, 2016 · Operations

Mastering Log Management: Step-by-Step ELK Stack Installation on CentOS

This guide walks you through the fundamentals of big‑data characteristics, the challenges of log management for small‑scale operations, and provides a detailed, step‑by‑step tutorial for installing and configuring the ELK stack (Elasticsearch, Logstash, Kibana) on a CentOS 6.6 server, including all required dependencies.

CentOSELKElasticsearch
0 likes · 15 min read
Mastering Log Management: Step-by-Step ELK Stack Installation on CentOS
Efficient Ops
Efficient Ops
Oct 27, 2015 · Operations

How to Build a Practical Monitoring System for Small and Medium Enterprises

An in‑depth guide walks readers through building a comprehensive monitoring system for small‑to‑medium enterprises, covering hardware, system, application, network, security, traffic analysis, business metrics, log aggregation, automation, visualization, and practical integration with tools like Zabbix, IPMI, ELK, and Smokeping.

AutomationOperationsSystem Monitoring
0 likes · 18 min read
How to Build a Practical Monitoring System for Small and Medium Enterprises
21CTO
21CTO
Sep 28, 2015 · Operations

Mastering Log Management: 16 Rules to Boost System Reliability

This article presents a comprehensive set of logging best‑practice rules—from defining log levels and classifications to using RequestIDs, monitoring alerts, and managing log size—aimed at improving system reliability, troubleshooting speed, and operational efficiency.

LoggingOperationsbest practices
0 likes · 23 min read
Mastering Log Management: 16 Rules to Boost System Reliability
MaGe Linux Operations
MaGe Linux Operations
May 5, 2015 · Operations

Master Logrotate: Automate Linux Log Management Efficiently

This guide explains how to use logrotate on Linux to automatically rotate, compress, and delete log files, covering installation, configuration examples for size‑based and time‑based rotation, and essential options to keep server storage under control.

Automationlog managementlogrotate
0 likes · 6 min read
Master Logrotate: Automate Linux Log Management Efficiently