Kubernetes Audit Log Analysis for Container Security

This article discusses the use of Kubernetes audit logs for detecting container security threats, particularly focusing on the CDK (Container Exploitation Toolkit) automated attack tool. The author, Zhang Zhi, a senior security engineer at Bilibili, provides a comprehensive analysis of how to monitor and detect various attack behaviors through audit log analysis.

The article begins by introducing the background of container security threats and the importance of audit logging in Kubernetes. It explains how audit logs provide a chronological record of activities affecting the system, helping administrators answer questions about what happened, when, who triggered it, why it occurred, where it was observed, where it originated from, and what consequences it will produce.

The author then details the process of enabling audit logging in Kubernetes, including configuration parameters for the API server and using kubeadm for setup. The main focus is on analyzing attack behaviors detected through audit logs, including information gathering, vulnerability exploitation, privilege escalation, and persistence mechanisms.

Key detection methods include monitoring specific log fields such as userAgent, responseStatus.code, requestURI, and various object references. The article provides numerous code examples showing actual audit log entries for different attack scenarios, including API server probing, namespace enumeration, API access testing, container escape attempts, network scanning, information theft, RBAC privilege escalation, and backdoor deployment.

The author also discusses detection of other security threats beyond CDK, such as CVE-2022-3172 K8s aggregation API SSRF vulnerability, use of non-compliant container images, and pod command execution. Practical implementation challenges and solutions are addressed, including handling large log volumes and inconsistent field types.

The article concludes with recommendations for comprehensive security practices, emphasizing the importance of baseline controls, admission policies, and runtime intrusion detection platforms to ensure cluster security and stability.