Tagged articles
Vision Transformers
13 articles · Page 1 of 1
The paper systematically studies Test‑Time Training (TTT) for vision, derives six design principles, and introduces ViT³—a pure TTT architecture that uses full‑batch internal training, a learning rate of 1.0, and lightweight SwiGLU‑Depthwise convolution modules, achieving state‑of‑the‑art linear‑complexity performance across classification, detection, segmentation and generation tasks.
Computer VisionLinear ComplexitySequence Modeling
0 likes · 14 min read

The paper introduces OrthoReg, a lightweight orthogonal regularization added during fine‑tuning that provably enforces weight orthogonality, thereby resolving conflicts in model merging and providing a theoretical explanation for the success of task arithmetic.
OrthoRegOrthogonal RegularizationTask Arithmetic
0 likes · 12 min read


AIWalkerMar 18, 2026 · Artificial Intelligence
The AAAI 2026 paper by Tsinghua’s Huang‑Gao team shows that modeling Vision‑Transformer attention as a Block‑Circulant matrix and computing it with FFT reduces the quadratic complexity to O(N log N), delivering up to seven‑fold real‑world speedups without sacrificing accuracy.
AAAI 2026Circulant MatricesComputer Vision
0 likes · 15 min read


SuanNiFeb 26, 2026 · Artificial Intelligence
BitDance’s new multimodal AI model achieves an 8.7‑fold inference acceleration using only 2.6 billion parameters, surpasses 14‑billion‑parameter state‑of‑the‑art architectures in image generation quality, and introduces binary visual tokens, a binary diffusion head, and next‑block diffusion for efficient parallel autoregressive prediction.
AIBinary TokenizationModel Efficiency
0 likes · 11 min read

The LaVin-DiT paper presents a large vision diffusion transformer that integrates a spatio‑temporal variational auto‑encoder, a joint diffusion transformer with full‑sequence joint attention, and 3D rotary position encoding to enable unified, efficient multi‑task generation for images and videos, and details its training via flow‑matching and experimental results.
3D RoPEComputer VisionJoint Diffusion Transformer
0 likes · 12 min read


AIWalkerJun 24, 2025 · Artificial Intelligence
The paper introduces Mamba-Adaptor, a plug‑and‑play module combining Adaptor‑T and Adaptor‑S to overcome causal computation, long‑range forgetting, and spatial modeling limits of visual Mamba models, delivering top‑ranked results on ImageNet and COCO across multiple downstream tasks.
AdaptorMambaState Space Model
0 likes · 25 min read

The CVPR 2025 awards spotlight groundbreaking research, honoring young scholars and top papers such as VGGT, Neural Inverse Rendering, and several honorable mentions, while summarizing each work's core contributions, methodologies, and potential impact on computer vision and related fields.
2025CVPRComputer Vision
0 likes · 13 min read

This article critically examines the MambaOut paper, analyzing whether state‑space‑based Mamba token mixers are necessary for vision tasks, presenting two hypotheses, describing the construction of MambaOut models without SSM, and reporting extensive ImageNet, COCO and ADE20K experiments that reveal when Mamba is beneficial.
MambaState Space ModelToken Mixer
0 likes · 17 min read


AIWalkerFeb 26, 2025 · Artificial Intelligence
The paper analytically identifies injectivity and local modeling as the two key factors causing the performance gap between linear and Softmax attention, proposes the InLine attention modifications to restore these properties, and demonstrates through extensive Vision Transformer experiments that the enhanced linear attention matches or surpasses Softmax while retaining linear computational cost.
Attention MechanismEfficient TransformersLinear attention
0 likes · 24 min read

This article introduces the four types of machine learning, explains self‑supervised learning, details generative and contrastive approaches, and provides an in‑depth overview of the MoCo series (V1‑V3), including their architectures, training strategies, and experimental results on document image classification and text‑line detection tasks.
Contrastive LearningMoCoVision Transformers
0 likes · 12 min read

The Twins series, a collaboration between Meituan and the University of Adelaide, introduces conditional positional encoding and spatially separable self‑attention to improve efficiency and performance of vision transformers, achieving state‑of‑the‑art results on ImageNet, ADE20K, COCO and high‑precision map segmentation.
ADE20KCOCOConditional Positional Encoding
0 likes · 20 min read

This article provides a detailed technical walkthrough of Masked Autoencoders (MAE) for computer vision, covering its BERT‑inspired masking strategy, asymmetric encoder‑decoder design, implementation specifics, experimental findings on mask ratios and decoder depth, and the resulting performance gains over supervised ViT models.
Computer VisionMAEMasked Modeling
0 likes · 11 min read


Code DAODec 8, 2021 · Artificial Intelligence
This article walks through the design of Compact Transformers, explaining scaled dot‑product self‑attention, positional embeddings, multi‑head attention, and Vision Transformer architecture, and provides full PyTorch code so readers can train lightweight CV and NLP classifiers on a single PC.
Compact TransformersPatch EmbeddingPositional Embedding
0 likes · 19 min read
