BestHub
Sign in
Tagged articles

local attention

2 articles · Page 1 of 1
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Aug 2, 2026 · Artificial Intelligence

A 35M-Parameter Model Trained on a Single GPU Claims Best Sub-100M Performance

Developer Harshal Singh released BarunLM-35M, a 35-million-parameter language model that fits on an ESP32-S3, achieves 41.01% average accuracy on nine zero-shot benchmarks—outperforming larger 160-M-parameter models—using a single H200 GPU, with novel alternating local/global attention and a learnable residual selector.

BarunLM-35MGPU traininglocal attention
0 likes · 6 min read
A 35M-Parameter Model Trained on a Single GPU Claims Best Sub-100M Performance
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Jul 24, 2023 · Artificial Intelligence

Understanding Slide-Transformer: An Efficient Local Attention Module for Vision Transformers

This article explains the Slide-Transformer paper, describing how the proposed Slide Attention replaces inefficient Im2Col‑based local attention with depthwise convolutions and a deformable shift module, achieving high efficiency, flexibility, and hardware‑agnostic performance for Vision Transformers.

Computer VisionDeep LearningDeformable Shift
0 likes · 13 min read
Understanding Slide-Transformer: An Efficient Local Attention Module for Vision Transformers