How Hierarchical Attention Boosts Dynamic Heterogeneous Graph Embedding

Introduction

Most graph embedding methods such as node2vec, GCN, GraphSAGE and GAT assume static graphs, but real‑world relationship graphs evolve over time. For example, a user's interest may shift from dresses to high‑heels, which cannot be explicitly modeled by static graph methods.

Dynamic Graph Representation Learning

Dynamic graph embedding learns both the current structure and its temporal evolution. Existing works like DynamicTriad and DySAT focus on dynamic homogeneous graphs. Inspired by DySAT and HAN, we propose DyHAN, a hierarchical‑attention dynamic heterogeneous graph embedding algorithm that outperforms current methods in offline evaluations.

Graph Construction

We build a heterogeneous dynamic graph from user behavior logs. Nodes are users and items; edge types include click, inquiry (AB), and order. Each day forms a time slice, with 10 days for training and the 11th day for testing. The resulting graph has 11 time slices, 2 node types, and 3 edge types.

Model Architecture

DyHAN consists of three attention layers: node‑level, edge‑level, and temporal‑level aggregation. Each layer can be replaced by alternative aggregators (e.g., mean‑pooling, LSTM). Multi‑head attention can be applied to increase expressiveness. The loss function is cross‑entropy with positive/negative samples selected from the last time slice.

Node‑level Attention

For each time slice and edge type, node embeddings are obtained by attending to the node itself (query) and its neighbors (keys), producing a representation that captures semantics under that edge type.

Edge‑level Attention

Edge‑type vectors for each node are aggregated, allowing the model to weigh more important edge types (e.g., order edges) higher.

Temporal‑level Attention

Node embeddings across time slices are combined using scaled dot‑product attention with a mask that prevents future information leakage.

Experiments

We evaluate DyHAN on edge‑prediction (link prediction) tasks using two public datasets and an internal Alibaba ICBU dataset. Baselines include static methods (DeepWalk, metapath2vec, GraphSAGE, GAT) and dynamic methods (DynamicTriad, DySAT). DyHAN consistently achieves higher accuracy.

Online Deployment

In the Alibaba International (ICBU) recommendation system, we replace the static GraphSAGE i2i model with a dynamic version based on DyHAN. Offline tests show a 10.9% increase in coverage compared to a 4.2% gain from the static model. Online A/B tests on detail‑page cross‑shop recommendation yield conversion rate improvements of up to 14.23%.

Conclusion

DyHAN introduces a hierarchical‑attention framework for dynamic heterogeneous graph embedding, achieving practical gains in Alibaba’s recommendation scenario. Future work includes reducing computational overhead of time‑slice processing and exploring more effective temporal fusion techniques.