A Brief Overview of Graph Neural Networks: GCN, GraphSAGE, GAT, GAE and DiffPool

Why Graph Neural Networks?

Traditional deep learning excels on structured data such as sequences and grids (e.g., text, images), but many real‑world systems—social networks, knowledge graphs, file systems—are non‑structured graphs. GNNs were introduced to model such irregular data, enabling breakthroughs in network analysis, recommendation, physical modeling, NLP, and combinatorial optimization.

What is a Graph Neural Network?

A GNN extends the basic multilayer perceptron by incorporating an adjacency matrix. The core operation multiplies the node feature matrix with a normalized adjacency matrix, applies a linear transformation, and a non‑linear activation (see Figure 3 in the source).

Classic GNN Models

1. Graph Convolutional Networks (GCN)

GCN is the pioneering GNN model that adapts convolution from image processing to graph data. It aggregates neighbor features, performs a linear transformation, and stacks multiple layers to capture K‑hop neighborhoods. Experiments on citation datasets (Cora, Citeseer, Pubmed, NELL) show significant accuracy gains over traditional methods.

GCN drawbacks: high memory consumption (requires the whole graph in memory) and the need for the full graph structure during training, which limits scalability and inductive capability.

2. Graph Sample and Aggregate (GraphSAGE)

GraphSAGE addresses GCN’s limitations by using a sampling‑and‑aggregation scheme that enables inductive learning. It samples a fixed number of neighbors, aggregates their embeddings (mean, LSTM, or pooling), and updates the target node’s embedding. This allows handling unseen nodes and large graphs.

Experimental results on citation, Reddit, and PPI datasets demonstrate clear improvements over baseline methods for both supervised and unsupervised tasks.

3. Graph Attention Networks (GAT)

GAT introduces masked self‑attention to weight neighbor contributions differently, similar to the Transformer’s attention mechanism. Multi‑head attention further enhances expressive power, and the model works for both transductive and inductive settings.

GAT achieves higher accuracy than traditional GNNs on several benchmark tasks.

Unsupervised Node Representation Learning

When labeled data is scarce, unsupervised methods such as Graph Auto‑Encoder (GAE) and its variational variant (VGAE) learn node embeddings by reconstructing graph structure. GAE uses a two‑layer GCN encoder and a simple reconstruction loss, while VGAE adds a KL‑divergence term.

Both models outperform traditional baselines on link‑prediction tasks across Cora, Citeseer, and Pubmed.

Graph Pooling

Pooling aggregates node embeddings to obtain a whole‑graph representation, essential for graph‑level classification. DiffPool introduces a differentiable pooling layer that learns a hierarchical assignment matrix, enabling end‑to‑end training with GNNs.

DiffPool achieves the best average performance among pooling methods on several graph classification benchmarks (ENZYMES, PROTEINS, D&D, REDDIT‑MULTI‑12K, COLLAB), though it incurs higher memory cost due to the assignment matrix.

Summary of Advantages and Limitations

GCN: simple and effective but memory‑intensive and requires full graph.

GraphSAGE: scalable, inductive, parameter‑efficient.

GAT: attention‑based weighting, fast parallel computation, works for both learning settings.

GAE/VGAE: unsupervised representation learning with strong link‑prediction performance.

DiffPool: hierarchical pooling with superior classification accuracy, limited by memory.

References

1. Graph Neural Networks: A Review of Methods and Applications (arXiv 2018). 2. A Comprehensive Survey on Graph Neural Networks (arXiv 2019). 3. Deep Learning on Graphs: A Survey (arXiv 2018). 4. GNNpapers GitHub list. 5. Semi‑Supervised Classification with Graph Convolutional Networks (ICLR 2017). 6‑12. Additional citations as listed in the original article.