From Math to ML: My Path Through Recommendation, Security, and AIOps

Preface

The author, originally trained in pure mathematics, decided to switch to machine learning after the 2016 AlphaGo breakthrough. Motivated by the rapid rise of AI in public awareness, the author began learning ML despite having no prior exposure.

2015: Trying to Switch

After graduating in 2015, the author joined Tencent and worked on a recommendation project built on a large‑scale data cluster without documentation or front‑end. This experience led to learning basic Linux commands, SQL, and Hive. The author wrote an introductory article on Hive and explored logistic regression, cross‑validation, and feature engineering techniques such as inner and outer products, standardization, discretization, and binarization. Model evaluation relied on inspecting learned weights, Pearson correlation, and KL divergence.

The author also shared articles on cross‑validation, feature engineering, and KL divergence, emphasizing the importance of hands‑on practice.

2016: From Zero to One

In early 2016 the author expanded recommendation work to include Item‑based Collaborative Filtering and heat‑diffusion algorithms for personalized tab recommendations. Reading Zhou Zhihua’s textbook "Machine Learning" and the companion "Machine Learning in Action" deepened theoretical understanding.

A security‑focused ML project was launched, requiring >99% accuracy. Existing recommendation pipelines (logistic regression, ItemCF, heat diffusion) were insufficient, prompting the design of a new framework tailored to security scenarios.

The author investigated unsupervised and supervised anomaly detection methods, eventually publishing a series of articles on point‑anomaly detection.

2017: Re‑energizing

The author explored quantum computing, publishing two introductory articles, and then turned to intelligent operations (AIOps). Challenges included heavy legacy baggage, shortage of AIOps talent, and lack of mature frameworks.

Key projects involved multi‑dimensional root‑cause analysis for a monitoring system and time‑series anomaly detection for millions of KPI curves. After evaluating ARIMA, RNN, LSTM, and Prophet, the author proposed a two‑stage solution: an unsupervised filter followed by a supervised model, and later an ensemble that treats predictions of multiple models as features for a generic detector.

Looking Ahead

The author reflects on the painful data‑quality checks, the joy of meeting business metrics, and the importance of solid planning and continuous learning for anyone transitioning into ML. Emphasis is placed on building AIOps capabilities to achieve “million‑curve‑one‑person” anomaly detection and on fostering long‑term goals within teams.