How Visualizing Deep CTR Models Turns Black Boxes into Insightful Tools

Background

Deep learning has achieved remarkable progress but remains a "black box"; recent efforts aim to make its mechanisms more transparent for safety, reliability, and optimization. Visualizing models helps humans understand and evaluate neural networks, especially in e‑commerce advertising where CTR prediction is critical.

To address this, the DeepInsight platform was built for large‑scale industrial deep learning applications, providing data extraction, multi‑dimensional visualization, real‑time analysis, and re‑modeling capabilities.

Platform Overview

DeepInsight is a distributed micro‑service system composed of a front‑end web UI, back‑end services, and deep‑learning components. It supports TensorFlow and MXNet, multiple learning paradigms (multi‑task, transfer, reinforcement, GAN, model ensemble), and offers lifecycle management for training tasks, enabling one‑stop visual evaluation and feedback loops.

Algorithm Experiments

A representative simple GwEN‑style CTR model is used: sparse feature IDs are embedded, summed per feature group, concatenated, and fed into four fully‑connected ReLU layers, with a sigmoid output for predicted CTR (PCTR). Dynamic data extraction captures internal states at various training stages for visualization.

Generalization Effect and Neuron State Fluctuation

Neuron state variability, driven by input differences, reflects model sensitivity; excessive sensitivity leads to over‑fitting and reduced generalization. Visualizations show that before over‑fitting, neuron fluctuations are stable across train/test sets, while over‑fitting causes a sharp increase in fluctuation, especially on training data.

The average fluctuation across a hidden layer correlates with AUC changes, providing a label‑free metric to detect over‑fitting.

Feature Influence

Unlike linear models, deep networks automatically learn non‑linear feature interactions, yet raw feature quality still matters. Gradient‑based attribution computes the sensitivity of the PCTR output to each feature group, revealing that over‑fitted models become overly dependent on a few high‑cardinality features (e.g., user ID).

Utility and Representation of Hidden Layers

t‑SNE projections of hidden‑layer outputs show that, despite high noise, click samples form clusters, with the third layer offering the most discriminative representation; the fourth layer adds little benefit, suggesting it can be omitted without performance loss.

Re‑modeling Hidden‑Layer Representations

Using Alain and Bengio's probing method, linear classifiers are trained on hidden‑layer representations to predict clicks. Probe performance improves from layer 1 to 3, confirming that deeper layers encode more useful information, while layer 4 provides no additional gain.

Conclusion

By visualizing and interpreting deep CTR models in e‑commerce advertising, DeepInsight opens the black box, enabling deeper understanding of model states, feature impacts, and hidden‑layer utilities, thereby supporting both algorithm research and practical business deployment.