This article explores the everyday challenges of operations teams handling system, access, runtime, error, and business logs, outlines the pain points of log collection and standardization, and provides a comprehensive guide to implementing the ELK (Elastic) stack—including Elasticsearch, Logstash, and Kibana—for effective monitoring and analysis.
This article introduces Meitu’s self‑developed log collection system Arachnia, explaining why a custom solution was needed for massive server‑side user‑behavior logs, the key requirements such as reliability and real‑time throughput, and the core architectural mechanisms that address those challenges.
This article explains why service monitoring is essential for large‑scale microservice environments, outlines design principles, core monitoring components, dependency mapping, call‑chain analysis, capacity planning, root‑cause analysis, and presents a practical technical architecture for implementing robust monitoring solutions.
This article explains how LogHub enables real-time log collection and unified management for an e‑commerce takeout platform, covering operational challenges, logstore configuration, user promotion tracking, server and client logging methods, and network access options.
The article presents the architecture, component design, optimizations, and reliability guarantees of Bilibili's Lancer log collection system, a Flume‑based distributed pipeline that handles both real‑time and offline data streams for billions of events daily.
This guide walks you through installing and configuring the ELK stack (Elasticsearch, Logstash, Kibana), setting up the required environment, creating Logback and Logstash configurations, and integrating them with a Spring MVC application to collect and visualize Java logs in Kibana.
Beats are lightweight data shippers in the ELK Stack, and this guide details the purpose, key features, and integration of Filebeat for log collection, Metricbeat for system and service metrics, Packetbeat for network traffic analysis, Winlogbeat for Windows event logs, and Heartbeat for service uptime monitoring.
This article reviews major open‑source log collection tools—Chukwa, Scribe, Flume, Logstash, Kafka, and TT—examining their architectures, strengths, and limitations to help engineers select the most suitable solution for high‑volume, low‑latency data pipelines.
This article presents a practical architecture for centrally collecting dispersed logs from Docker‑based services in a distributed environment by leveraging Flume NG as a non‑intrusive log agent, Kafka as a high‑throughput message bus, and custom sinks to partition logs by service, module, and day.
This article explains how to capture Docker container logs, discusses the challenges of multi‑line log correlation, and presents two approaches—client‑side parsing and server‑side parsing—along with a concrete Flume customization using a DockerLog Java bean.
This guide explains how Oracle's Trace File Analyzer (TFA) simplifies and accelerates log collection across single‑node and RAC databases, covering supported platforms, installation details, workflow, command usage, analysis features, and practical tips for permissions and configuration.
This article discusses various log collection and analysis architectures, comparing solutions such as Flume‑Kafka‑Storm pipelines, Sentry, MongoDB, ELK stack, and Hadoop, and shares practical experiences, advantages, drawbacks, and deployment tips from multiple engineers.
This article provides a comprehensive overview of App Engine as a PaaS solution, analyzes leading platforms such as CloudFoundry and SAE, outlines architectural requirements, and presents practical implementation guidance using open‑source tools like Nginx, Scribe, and Storm for elastic, scalable cloud services.
