This article explains how to extract valuable performance and traffic insights from Nginx access logs using shell commands and awk, covering request volume, peak rates, bandwidth usage, slow‑query detection, URL normalization, and practical optimization recommendations for web operations.
This article explains how to apply big‑data concepts and platforms to operations monitoring, covering data sources, metric extraction from logs, architectural design with Flume, Spark Streaming and HBase, implementation steps, and the resulting benefits for scalability and rapid metric development.
This guide explains essential Linux audit commands, log‑viewing techniques, user and process inspection methods, and additional checks such as file integrity, network monitoring, cron jobs, and backdoor detection, providing a comprehensive toolkit for system security and operations.
This article summarizes a 2016 Global Operations conference talk that explains the concept of IT Operations Analytics (ITOA), its four data sources, the evolution of log management from databases to real‑time search engines, and real‑world case studies demonstrating how fast, large‑scale log analysis improves monitoring, security, and business insight.
This article explains why log analysis platforms are essential as systems grow, outlines the benefits of centralized logging, presents Ctrip’s real‑world requirements and challenges, and introduces the ELK stack as a scalable solution for collecting, storing, and visualizing massive log data.
This guide explains how to use the lightweight ngxtop tool to monitor Nginx in real time, showing frequent requests, high‑traffic IPs, and providing useful command examples for filtering logs, ordering results, and installing the utility via pip.
On February 10, 2016 a provincial accounting database experienced severe daily‑batch delays despite other applications running normally; a senior DBA collected alert logs, AWR snapshots, ASH dumps, and OSW metrics, uncovered log‑file‑sync and redo‑IO degradation, increased rollbacks, and a faulty SAN link, pinpointing the root cause.
This article shares the author's practical experience building a real‑time log analysis platform at Sina, covering service scope, ELK architecture, performance optimizations, usability improvements, new features, common pitfalls, and a concise Q&A for engineers handling massive log streams.
This article explores how logs serve as vital diagnostic tools in operations, detailing their definition, practical uses for performance and security monitoring, step‑by‑step analysis techniques, and recommendations for free and commercial log analysis solutions.
Learn how to parse web server access logs with AWK, aggregate response times per URL, calculate average latency for each request, and output the results, enabling you to pinpoint long‑running requests that degrade system performance.
Learn three powerful shell commands that reveal the most active IPs, the most requested URLs, and the largest files in an Apache access_log, helping you quickly detect anomalies, optimize resources, and improve site performance.
The article outlines five subtle indicators of a web application breach—abnormal behavior, irregular logs, unexpected processes or users, file modifications, and warning messages—while offering practical monitoring and remediation steps to help security teams detect and mitigate attacks early.
The article explains why user‑behavior data is often ignored by UED teams, outlines a three‑stage collection‑analysis‑reporting pipeline, and shows how JavaScript running on V8/Node.js can empower front‑end developers to process massive logs, reducing reliance on separate BI resources.
Learn essential log analysis techniques using command‑line tools like grep, cut, and awk, explore advanced filtering with rsyslog and Grok, and discover how log management systems can automate parsing and accelerate troubleshooting of Linux syslog data.
Learn how to use the open‑source Mtools suite to inspect MongoDB logs, extract slow queries, generate performance charts, and overlay system metrics, enabling effective regular health checks, troubleshooting, and optimization for database administrators.
A recent study of five major open‑source distributed systems reveals that most failures can be triggered and reproduced with simple, multi‑event tests, highlighting the importance of systematic testing, deterministic error handling, and concise logging for reliable system operation.
When troubleshooting, you often need to isolate log entries from a precise time window; instead of cumbersome regular expressions, this guide shows how to efficiently capture nginx access logs between two timestamps with a simple awk command, illustrated with an example and a screenshot.
