How the MPCT Multiscale Point Cloud Transformer Boosts 3D Classification Accuracy

Abstract

Self‑attention networks are permutation‑invariant and thus suitable for unordered 3D point sets, but naïve attention incurs quadratic complexity and memory use. The authors introduce a position‑encoding that linearizes attention, yielding a multiscale point‑cloud transformer (MPCT) whose computational cost is O(N·D) instead of O(N²). MPCT achieves state‑of‑the‑art results on standard benchmarks, e.g., 94.2% classification accuracy on ModelNet40 and 84.9% on ScanObjectNN.

Methodology

Transformer Preliminaries

Given an input point set X∈ℝ^{N×D}, linear projections produce query Q, key K, and value V matrices. Self‑attention is computed as softmax((QKᵀ)/√d)·V, which is permutation‑invariant.

MPCT Architecture

The backbone consists of four stages. Each stage performs:

Sampling (farthest point sampling) with a configurable down‑sampling ratio.

Grouping of K nearest neighbors.

Linear‑Batch‑ReNorm (LBR) layers for normalization and activation.

Residual connections that add the original features to the attention‑enhanced output before passing to the next stage.

Feature dimensions and sampling rates are configurable, allowing the network to adapt to different point‑cloud densities.

Point Enhancement

For every point, its two nearest neighbors are located, forming a triangle. A geometric descriptor (e.g., edge lengths, angles) is computed from this triangle and fed into a learnable point encoder, producing enriched point features that capture local shape information.

Neighborhood Enhancement

Each point’s K nearest neighbors are encoded into feature vectors. These vectors are duplicated to form a matrix that represents the local geometric relations. A position‑encoding module processes the raw coordinates to extract higher‑level contextual cues, which are concatenated with the neighbor features.

Point‑Cloud Transformer Layer

Queries and keys are added to the positional‑encoding matrix, then pairwise subtraction yields an attention‑weight matrix A_{ij}= (Q_i - K_j). A bias‑free linear layer maps A to scalar scores, followed by row‑wise softmax normalization. The weighted sum of value vectors produces the self‑attention feature F_{att}=softmax(A)·V. The output is F_{out}=LBR(F_{att}+F_{in}), where F_{in} is the input feature.

Multiscale Feature Fusion

Four sampling‑and‑grouping (SG) layers progressively increase the receptive field:

Each SG layer samples points via farthest point sampling (FPS).

Local neighborhoods are aggregated using the encoded neighbor features.

Semantic features (from neighbor aggregation) are concatenated with geometric features (from point enhancement) and fed into the transformer layer.

The fused multiscale features are then passed to downstream heads for classification or segmentation.

Experiments

Extensive evaluation on ModelNet40, ScanObjectNN, and additional benchmarks demonstrates the superiority of MPCT. Notable results include:

ModelNet40 classification accuracy: 94.2%

ScanObjectNN classification accuracy: 84.9%

Figures below illustrate the architecture, point‑enhancement process, transformer layer design, and quantitative results.