Bypassing BPTT: MIT’s SMT Puts RNNs on the Parallel Training Path

The article reviews MIT’s Supervised Memory Training (SMT) and its DAgger extension (DMT), which replace traditional back‑propagation through time with a Transformer‑based teacher, enabling one‑step memory supervision for RNNs, achieving parallel‑friendly training and superior long‑sequence performance on synthetic benchmarks, TinyStories and pixel‑wise image generation.

BPTTDMTRNN

0 likes · 10 min read

Bypassing BPTT: MIT’s SMT Puts RNNs on the Parallel Training Path

Alibaba Cloud Big Data AI Platform

Oct 8, 2023 · Artificial Intelligence

Why the Scale‑Aware Modulation Transformer Outperforms CNNs and Vision Transformers with Fewer Parameters

The Scale‑Aware Modulation Transformer (SMT) introduces a lightweight SAM module and an Evolutionary Hybrid Network that together achieve higher accuracy on ImageNet, COCO, and ADE20K while using significantly fewer parameters and FLOPs than existing CNN and Transformer baselines.

SMTScale‑Aware ModulationSemantic Segmentation

0 likes · 12 min read

Why the Scale‑Aware Modulation Transformer Outperforms CNNs and Vision Transformers with Fewer Parameters

Network Intelligence Research Center (NIRC)

Aug 15, 2023 · Artificial Intelligence

Neural Networks for Rapid Network Configuration: A Concise Overview

The article presents a neural‑algorithmic reasoning approach that replaces slow SMT‑based network configuration tools with a graph‑neural‑network model, describing dataset creation, model architecture, and experiments that show 20‑to‑490× speedups while maintaining over 92% configuration consistency on large topologies.

Graph Neural NetworkNetwork ConfigurationNetwork Synthesis

0 likes · 5 min read

Neural Networks for Rapid Network Configuration: A Concise Overview