Resource Monitoring

FunTester

May 28, 2025 · Cloud Native

Extending Automated Thread Dumps: Log Collection, Resource Monitoring, Chaos Engineering, Performance Analysis, and Environment Cleanup

The article explores how automated thread dumps can be expanded into multiple testing scenarios—including log collection, resource monitoring, fault injection, performance result analysis, and environment cleanup—by leveraging Kubernetes APIs, Prometheus, Chaos Mesh, and scripting tools to improve efficiency, observability, and system resilience.

Chaos EngineeringKubernetesLog Collection

0 likes · 9 min read

Extending Automated Thread Dumps: Log Collection, Resource Monitoring, Chaos Engineering, Performance Analysis, and Environment Cleanup

OPPO Kernel Craftsman

Jan 3, 2020 · Operations

Understanding Linux PSI: Pressure Stall Information for System Resource Monitoring

Pressure Stall Information (PSI) is a Linux kernel feature that measures real‑time CPU, memory, and I/O pressure by tracking task wait times, offering finer granularity than load average or vmpressure, and enabling more accurate scheduling, cgroup management, and out‑of‑memory handling.

CPU schedulingLinux KernelMemory Management

0 likes · 14 min read

Understanding Linux PSI: Pressure Stall Information for System Resource Monitoring

Youzan Coder

Feb 1, 2019 · Big Data

Design and Implementation of Log Parsing for a Big Data Offline Task Platform

The article describes a log‑parsing feature for Youzan’s big‑data offline platform that captures runtime logs from Hive, Spark, DataX, MapReduce and HBase jobs, categorizes scheduling types, extracts metrics such as read/write bytes, shuffle volume and GC time, and processes them in real time via a Filebeat‑Logstash‑Kafka‑Spark‑Streaming pipeline storing results in Redis for monitoring, optimization and resource‑usage ranking.

Resource MonitoringSpark StreamingYARN

0 likes · 7 min read

Design and Implementation of Log Parsing for a Big Data Offline Task Platform

Youzan Coder

Aug 3, 2018 · Big Data

Youzan Data Warehouse Metadata System: From Manual Tables to Metadata‑Driven Architecture

Youzan’s data‑warehouse metadata system evolved from manually maintained tables to an automated data dictionary and finally to a metadata‑driven architecture that automatically captures technical, business, and process metadata, visualizes lineage, tracks resource usage, manages synchronization rules and permissions, and now aims to improve novice usability with visual models and impact‑analysis tools.

Data WarehouseHiveResource Monitoring

0 likes · 11 min read

Youzan Data Warehouse Metadata System: From Manual Tables to Metadata‑Driven Architecture