0 views collected around this technical thread.
This guide explains Logstash’s Grok filter plugin, detailing how its 120 built‑in and custom patterns transform unstructured logs—such as Apache, MySQL, or HiveServer2—into structured fields through named regex captures, supporting type conversion, cleaning, debugging, and efficient ETL for analysis and monitoring.
This article explains how to implement online traffic replay for Python automated testing by parsing logs, storing request data in a local MySQL database, analyzing user information with data‑analysis libraries, and locally persisting URL, method, headers, and body details using JSON files, complete with sample code.
This article explains how Python can automate common operations tasks—remote command execution, log parsing, system monitoring with alerts, bulk software deployment, and backup/restore—providing code examples for each and highlighting additional tools that help sysadmins improve efficiency and reduce errors.
This guide demonstrates how to configure Nginx log formatting, write Grok patterns, and set up Logstash to ingest Nginx access logs into Elasticsearch for visualization in Kibana, covering step‑by‑step configuration and regex extraction techniques.
This article explains why standardized log parsing is crucial for security analytics, outlines key parsing concepts, compares pre‑ and post‑parsing approaches, discusses flexible custom parsing methods, and offers practical guidance to improve accuracy and efficiency in large‑scale security environments.
The article describes a log‑parsing feature for Youzan’s big‑data offline platform that captures runtime logs from Hive, Spark, DataX, MapReduce and HBase jobs, categorizes scheduling types, extracts metrics such as read/write bytes, shuffle volume and GC time, and processes them in real time via a Filebeat‑Logstash‑Kafka‑Spark‑Streaming pipeline storing results in Redis for monitoring, optimization and resource‑usage ranking.
Xtrabackup is an open-source hot backup tool for InnoDB and XtraDB databases, offering non-blocking backups with features like fast backup speeds, reliable physical backups, and efficient disk space usage. The recovery process involves complex log parsing and page flushing mechanisms, which can be optimized to improve performance, especially for large datasets.