From Mathematics to Machine Learning: A Personal Journey Through Recommendation, Security, and AIOps

Introduction: The author, originally trained in mathematics, shares the pitfalls and detours encountered while transitioning to machine learning, aiming to provide a concise guide for newcomers.

Background: After graduating in 2015, the author joined Tencent and began working on machine‑learning‑related projects, despite having limited exposure to the field during graduate studies.

2015 – Early Projects: The first recommendation project involved a large‑scale data‑driven system without documentation or front‑end UI. The author learned basic Linux commands and SQL, deepening knowledge through books such as *SQL Fundamentals* and *HIVE Programming Guide*. Feature engineering concepts (inner product, outer product, Cartesian product, normalization, discretization, binarization) and model evaluation (weights, Pearson correlation, KL divergence) were explored, with articles on cross‑validation and feature engineering published on WeChat.

Algorithms Used: Logistic Regression was the initial offline model, later complemented by the online FTRL algorithm (Follow‑the‑Regularized‑Leader). The importance of data quality and rigorous data verification across offline, online, and product layers was emphasized.

2016 – Security‑Focused ML: The author helped establish a security‑oriented ML project, requiring higher accuracy (>99%). Traditional recommendation algorithms (Logistic Regression, ItemCF, heat‑propagation) were insufficient, prompting the design of a new framework. Research on unsupervised + supervised + manual labeling pipelines led to a series of articles on anomaly‑point detection.

Game AI Exploration: In late 2016, the author investigated reinforcement learning and deep learning, building a simple DQN to create a game AI, and authored articles on reinforcement learning and deep learning.

2017 – Quantum Computing & AIOps: The author studied quantum computing fundamentals and quantum‑ML concepts, publishing introductory articles. Later, the focus shifted to intelligent operations (AIOps), addressing challenges such as heavy legacy baggage, talent shortage, and lack of mature frameworks. Projects included multi‑dimensional root‑cause analysis, time‑series anomaly detection for millions of KPI curves, and a hybrid unsupervised‑supervised solution that dramatically reduced manual threshold configuration.

Methodology for Time‑Series Anomaly Detection: Various models (ARIMA, RNN/LSTM, Prophet) were evaluated, but a unified ensemble approach—using multiple model outputs as features for a meta‑learner—proved effective for heterogeneous series.

Future Outlook: The author believes machine learning will continue to permeate operations, driving the transition from manual DevOps to AI‑augmented AIOps, and hopes the shared experiences will help others transition into the field.