Skip to content
Tagged articles

Multimodal Pretraining

3 articles · Page 1 of 1
Machine Heart
Machine Heart
Apr 27, 2026 · Artificial Intelligence

Google DeepMind Open‑Sources TIPSv2: State‑of‑the‑Art Patch‑Text Alignment at CVPR 2026

The DeepMind team unveils TIPSv2, a vision‑language pre‑training model that dramatically improves patch‑level image‑text alignment through iBOT++, Head‑only EMA, and multi‑granularity captions, achieving record‑breaking results on nine tasks across twenty datasets while remaining fully open‑source.

Computer VisionDeepMindMultimodal Pretraining
0 likes · 12 min read
Google DeepMind Open‑Sources TIPSv2: State‑of‑the‑Art Patch‑Text Alignment at CVPR 2026
AntTech
AntTech
Jul 31, 2023 · Artificial Intelligence

LayoutMask: Enhancing Text-Layout Interaction in Multi-modal Pre-training for Document Understanding

LayoutMask introduces a novel multi-modal pre‑training model that replaces global 1D position with local 1D position and adds Whole Word Masking, Layout‑Aware Masking, and Masked Position Modeling, achieving state‑of‑the‑art results on various visually‑rich document understanding tasks.

AIDocument UnderstandingMultimodal Pretraining
0 likes · 15 min read
LayoutMask: Enhancing Text-Layout Interaction in Multi-modal Pre-training for Document Understanding
DataFunTalk
DataFunTalk
Mar 20, 2020 · Artificial Intelligence

UNITER: Unified Image‑Text Representation Learning for Vision‑Language Tasks

This article introduces UNITER, a unified image‑text representation learning framework pretrained on four large multimodal datasets, describes its three pretraining tasks (MLM, ITM, MRM), details model architecture, training optimizations, and evaluates performance across six vision‑language downstream tasks, achieving state‑of‑the‑art results.

AIITMMLM
0 likes · 11 min read
UNITER: Unified Image‑Text Representation Learning for Vision‑Language Tasks