Tagged articles
vision-language
43 articles · Page 1 of 1
Researchers from Northwestern Polytechnical University and HKUST propose Dynamic Gated Pathways (DGP), a method that lets each LLM layer dynamically attend to all ViT layers, improving multimodal benchmarks by learning per-layer, per-sample routing weights instead of using a fixed visual projection.
DGPDynamic Gated PathwaysLLM
0 likes · 18 min read

In the first half of 2026, 360 AI Institute saw six papers accepted at ICLR, CVPR, ICML and ECCV, covering multimodal understanding, agents, image generation and editing, while Chinese contributions reached 43.7% of ICLR submissions, highlighting a global move toward more precise and controllable artificial intelligence.
AI researchAgentsGenerative AI
0 likes · 9 min read

The paper introduces Counterfactual Policy Optimization (CFPO), a training‑time framework that inserts causal consistency constraints into multimodal reinforcement learning, forcing vision‑language models to rely on essential visual evidence and achieving consistent accuracy gains across real‑world and math‑centric benchmarks.
ICML2026causal consistencycounterfactual
0 likes · 19 min read

The paper introduces NEO‑ov, a native vision‑language model that discards external visual encoders, feeding raw pixels directly into a unified transformer, and demonstrates competitive performance on image, multi‑image, and video tasks—including fine‑grained perception and spatial reasoning—while outlining its three‑stage training pipeline and current limitations.
Qwenbenchmarkmultimodal
0 likes · 13 min read

The BARD-VL framework bridges pretrained autoregressive vision‑language models to diffusion‑based VLMs, preserving or surpassing original performance while boosting decoding throughput up to three times, through progressive block merging, stage‑wise diffusion distillation, and engineering optimizations validated on multiple benchmarks.
BARD-VLEfficiencybenchmark
0 likes · 9 min read


ArcThinkApr 29, 2026 · Artificial Intelligence
The article dissects DeepSeek V4's newly released vision mode, explains its mounted visual‑language architecture, compares its multimodal capabilities and costs against GPT‑5.5, Gemini 3 and Claude Opus 4.7, and outlines a roadmap from image understanding to native multimodal AI.
AIDeepSeekbenchmark
0 likes · 15 min read


SuanNiApr 29, 2026 · Artificial Intelligence
SenseNova U1, an open‑source multimodal model from SenseTime, replaces traditional visual encoders and VAEs with a native NEO‑unify architecture, delivering near‑lossless pixel‑level fidelity, a mixed‑of‑Transformer backbone, and unified training objectives that achieve SOTA performance on diverse vision‑language benchmarks while running efficiently on multiple Chinese chips.
NEO-unifySenseNova U1benchmark
0 likes · 9 min read

The DeepMind team unveils TIPSv2, a vision‑language pre‑training model that dramatically improves patch‑level image‑text alignment through iBOT++, Head‑only EMA, and multi‑granularity captions, achieving record‑breaking results on nine tasks across twenty datasets while remaining fully open‑source.
Computer VisionDeepMindMultimodal Pretraining
0 likes · 12 min read

This guide shows how to fine‑tune Qwen3.5 models—from 0.8B to 122B—using Unsloth Studio or pure code, covering text SFT, vision fine‑tuning, MoE models, reinforcement‑learning (GRPO), extensive GGUF quantization benchmarks, hardware requirements, export formats, and deployment tips.
LLMQwen3.5Unsloth
0 likes · 12 min read

The article introduces the Keyframe‑Chaining VLA (KC‑VLA) framework, which replaces dense video sampling with semantic keyframe linking to provide robots with global temporal awareness, presents a new long‑term memory benchmark, and demonstrates superior performance in both simulation and real‑world robotic experiments.
AIKeyframe ChainingLong-term Memory
0 likes · 9 min read

The article analyzes why text models naturally combine understanding and generation, explains the fundamental conflicts that prevent images from sharing the same tokenization, and details LongCat-Next’s discrete autoregressive approach—using SAE visual encoders, residual vector quantization, and a unified LLM backbone—to achieve a single model that can both comprehend and create multimodal content.
LongCat-NextMultimodal ModelsRVQ
0 likes · 21 min read

LongCat‑Next, Meituan’s new 3‑billion‑parameter foundation model, adopts a pure‑discrete DiNA architecture with next‑token prediction, converting vision, audio and text into unified tokens; it surpasses same‑size multimodal models on OmniDocBench‑EN, CharXivRQ and SWE‑Bench, avoids catastrophic forgetting, and introduces dNaViT, RVQ compression and a dual‑path detokenizer for high‑fidelity generation.
DiNALongCat-NextSWE-bench
0 likes · 10 min read

This article introduces Render‑of‑Thought (RoT), a novel paradigm that compresses chain‑of‑thought reasoning into visual embeddings using frozen vision encoders, achieving 3‑4× token reduction, faster inference, and improved interpretability while requiring minimal pre‑training.
Inference Optimizationchain-of-thoughtlatent space
0 likes · 12 min read

This article explains CLIP’s dual‑encoder architecture, contrastive training, and zero‑shot inference, then demonstrates its use through image‑text matching and CIFAR‑10 classification experiments with code examples, highlighting strengths and limitations such as resolution mismatch.
CLIPContrastive LearningImage-Text Matching
0 likes · 11 min read

DeepSeek-OCR 2 introduces a novel dual‑stream (bidirectional + causal) attention architecture that replaces fixed raster scanning, leverages a Qwen2‑0.5B encoder, and achieves state‑of‑the‑art accuracy on OmniDocBench while reducing token budget and improving reading‑order consistency.
DeepEncoderDeepSeekDocument Understanding
0 likes · 8 min read

AAAI 2026 in Singapore showcased 23,680 submissions, highlighting breakthrough papers such as ReconVLA’s reconstructive vision‑language‑action model, LLM2CLIP’s language‑enhanced multimodal representation, a sheaflet‑based hypergraph neural network design, advances in description logic modeling, and a novel causal discovery method for dynamical systems.
AAAI 2026AI papersLLM
0 likes · 7 min read

Kwai has open‑sourced its new flagship multimodal model Keye‑VL‑671B‑A37B, which upgrades visual perception, cross‑modal alignment and complex reasoning, achieving top scores on image, video, and mathematical reasoning benchmarks while detailing its architecture, three‑stage pre‑training, post‑training strategies, and future multimodal agent plans.
deep learninglarge language modelmultimodal
0 likes · 10 min read

This article reviews the YOLO-Count model, a fully differentiable, open‑vocabulary object counting system that guides text‑to‑image generators to produce the exact number of objects specified in prompts, achieving state‑of‑the‑art results on both generic counting and controlled image synthesis tasks.
Generative AIObject CountingYOLO-Count
0 likes · 8 min read

Kwai's newly released Keye-VL-1.5-8B multimodal large language model dramatically improves visual, reasoning, and temporal understanding, achieving top scores on public video benchmarks and surpassing closed‑source models like GPT‑4o, while offering an open‑source release and detailed technical documentation.
benchmark performancemultimodal LLMopen-source
0 likes · 11 min read

The Kwai Keye team presents Thyme, a novel multimodal reasoning framework that lets large language models generate and safely execute Python code for image manipulation and complex calculations, achieving significant performance gains over existing vision‑language models across perception, reasoning, and hallucination‑reduction benchmarks.
AI researchcode generationlarge language model
0 likes · 12 min read


AIWalkerAug 6, 2025 · Artificial Intelligence
The article provides an in‑depth technical analysis of ByteDance’s 7‑billion‑parameter BAGEL model, detailing its MoT architecture, high‑quality interleaved multimodal pre‑training data, multi‑stage training strategy, emergent capabilities, and extensive benchmark results that show BAGEL matching or surpassing GPT‑4o on vision‑language tasks.
BAGELGPT-4o comparisonUnified Model
0 likes · 24 min read

dots.vlm1, the first open‑source multimodal large model from Xiaohongshu hi‑lab, combines a 1.2‑billion‑parameter NaViT visual encoder with DeepSeek V3 LLM, achieving near‑state‑of‑the‑art visual understanding and reasoning while remaining competitive on text tasks, and is available on GitHub and HuggingFace.
AIdeep-learninglarge-model
0 likes · 11 min read

This article introduces UI‑TARS, a multimodal visual model combined with the Model Context Protocol (MCP) to build next‑generation cross‑platform autonomous GUI agents, detailing its architecture, workflow, code examples, incremental inference, applications, challenges, and future research directions.
AIGUI Agentautomation
0 likes · 20 min read

Multimodal AI expands beyond single‑modal text, image, or audio by integrating diverse data modes—such as audio‑spectrograms, video, tables, and even tactile formats—enabling models like CLIP (2021) and Flamingo (2022) to achieve superior vision‑language understanding, generation, and retrieval capabilities.
CLIPFlamingodata modalities
0 likes · 10 min read

This article introduces OpenAI’s CLIP multimodal model, explains its architecture and contrastive training, details hardware and installation steps, and demonstrates a hands‑on zero‑shot image classification workflow that achieves 97% confidence on a cat image without any task‑specific fine‑tuning.
CLIPContrastive LearningPython
0 likes · 6 min read

DeepSeek‑VL2 is a state‑of‑the‑art multimodal model built on a Mixture‑of‑Experts architecture that combines a SigLIP‑L vision encoder with dynamic tiling, a two‑layer VL adaptor, and a DeepSeek‑MoE language model using Multi‑head Latent Attention, trained in three stages on diverse visual‑language and text data, and achieving strong results on benchmarks such as DocVQA and TextVQA, with full implementation and inference code available in PaddleMIX.
DeepSeek-VL2Multimodal ModelPaddleMIX
0 likes · 36 min read

The Q‑Eval‑100K dataset, comprising 100 k AIGC images and videos with separate visual‑quality and textual‑consistency annotations, powers the open‑source Q‑Eval‑Score framework that fine‑tunes multimodal models to deliver state‑of‑the‑art, scalable, and objective evaluation—including a “vague‑to‑specific” strategy for long prompts—surpassing existing benchmarks.
AIGCDatasetevaluation
0 likes · 9 min read

This article compiles recent AI research breakthroughs presented at AAAI 2025 and NeurIPS 2024, summarizing eight papers on multi‑condition image generation, mixed auto‑regressive models, hallucination mitigation in vision‑language models, quantized diffusion denoising, facial part swapping, language‑guided concept vectors, attribution consistency, and video virtual try‑on, with links to each work.
AAAI 2025AI researchGenerative Models
0 likes · 13 min read


21CTODec 4, 2024 · Artificial Intelligence
Physical Intelligence's new Pi-zero model, built on a vision‑language foundation and fine‑tuned with extensive robot data, outperforms prior baselines across multiple tasks, showcasing the promise of large multimodal foundation models for flexible, robust robot control.
AIPi-zerofoundation models
0 likes · 6 min read

Professor Xie's team at Shanghai Jiao Tong University reports rapid progress in AI for Science, detailing multimodal medical AI models, large open datasets, language and vision‑language models, and knowledge‑enhanced representations that outperform existing baselines across multiple benchmarks.
Multimodal ModelsOpen Datasetsknowledge graphs
0 likes · 14 min read

This article reviews recent advances in multimodal large language models, covering their background, architectural components, training strategies, application scenarios, evaluation benchmarks, team research on hallucination mitigation and long‑video understanding, and outlines promising future research directions.
evaluation benchmarksfuture researchmodel architecture
0 likes · 15 min read

This article presents 360's comprehensive exploration of image‑text multimodal large models, covering background concepts, research routes, three generations of model development, proprietary architectures like SEEChat, 360VL and Inner‑Adaptor, and real‑world AI applications across various products and services.
AI applicationsmodel architecturevision-language
0 likes · 19 min read

The article introduces 360VL, an open‑source multimodal large language model built on Llama‑3‑70B, describes its novel C‑abs bridge architecture for high‑resolution visual understanding, outlines the two‑stage training with bilingual data, and presents benchmark results showing superior performance over prior LMMs.
AI researchlarge language modelllama3
0 likes · 8 min read


21CTOMay 14, 2024 · Artificial Intelligence
OpenAI’s latest flagship model GPT‑4o combines text, audio, image and video processing in a single, faster, cheaper multimodal system that delivers near‑human response times, expanded API access, and new safety measures, reshaping how developers and users interact with AI.
AI ModelGPT-4oOpenAI
0 likes · 10 min read

This article reviews Tongyi Lab's work on the OFA framework for generative multimodal pretraining and the ONE-PEACE model for unified multimodal representation learning, detailing their architectures, training strategies, experimental results across vision‑language and audio tasks, and future research directions.
OFAONE-PEACEPretraining
0 likes · 15 min read

This article presents MiniGPT-4, a multimodal system that combines a frozen visual encoder (Q‑Former + ViT) with an open‑source large language model (Vicuna), describes its motivation, training pipeline, demo capabilities, observed limitations, and includes a brief Q&A session.
AI researchImage CaptioningMiniGPT-4
0 likes · 15 min read

mPLUG-Owl is a modular multimodal dialogue large model from Alibaba DAMO Academy that builds on the mPLUG series, offering advanced image, video, OCR, and multilingual capabilities, with extensive evaluations showing superior performance over MiniGPT‑4, LLaVA, and other multimodal LLMs across various tasks.
evaluationmPLUG-Owlmultimodal AI
0 likes · 17 min read

The ACL 2023 paper introduces FashionKLIP, an e‑commerce visual‑language model enhanced by a multimodal concept knowledge graph, detailing its automated knowledge graph construction, dual‑stream training strategy, and superior performance on FashionGen retrieval benchmarks compared to state‑of‑the‑art methods.
FashionKLIPMultimodal Retrievale-commerce
0 likes · 10 min read

This article reviews the rapid development of multimodal attribute-level sentiment analysis on social media, outlining its background, defining four core sub‑tasks, summarizing representative recent models—including unified multimodal transformers, coarse‑to‑fine image‑target matching, and vision‑language pre‑training—and discussing experimental results and future research directions.
NLPaspect based sentimentdeep learning
0 likes · 21 min read

The article introduces the GIT image‑to‑text (image captioning) model, explains its transformer‑based architecture, showcases multiple example outputs, discusses training details, compares its performance with Flamingo and COCO, and highlights its applicability to tasks such as VQA, video captioning, and image classification.
GIT modelImage Captioningmodel comparison
0 likes · 12 min read

This article examines the CoCa model, explaining how it extends CLIP with image captioning by combining contrastive and generative objectives, detailing its architecture, training tricks, and performance gains on ImageNet and zero‑shot benchmarks.
CoCaImage Captioningvision-language
0 likes · 7 min read

GEN‑VLKT introduces a Guided‑Embedding Network with position‑ and instance‑guided embeddings to remove costly post‑processing and leverages CLIP‑based visual‑linguistic knowledge transfer for interaction understanding, achieving state‑of‑the‑art HOI detection performance and zero‑shot capability, now deployed in Alibaba’s Taobao services.
CLIPHOI detectionTransformer
0 likes · 7 min read

This article introduces UNITER, a unified image‑text representation learning framework pretrained on four large multimodal datasets, describes its three pretraining tasks (MLM, ITM, MRM), details model architecture, training optimizations, and evaluates performance across six vision‑language downstream tasks, achieving state‑of‑the‑art results.
AIITMMLM
0 likes · 11 min read
