Tagged articles
multimodal learning
89 articles · Page 1 of 1
Professor Li Haizhou's ACM MM 2026 keynote explores how machines can actively listen like humans by combining auditory, visual, and brain signals for target speaker extraction, demonstrating systems that use facial cues, body gestures, EEG decoding, and speech large models to isolate and understand a specific speaker in noisy multi-talker environments.
EEG decodingactive listeningbrain-computer interface
0 likes · 15 min read

Tsinghua graduate Xu Hanwen will join UPenn as assistant professor in 2027; his Nature-published Prov-GigaPath model, trained on 1.3 billion pathology images, achieved top performance on 25 of 26 tasks, surpassed 2 million Hugging Face downloads, and entered real-time clinical deployment in 2025.
AI in healthcareBiomedCLIPGigaTIME
0 likes · 5 min read

Researchers from NTU, SJTU, and UTokyo propose UMM-Reflection, enabling unified multimodal models to self-correct generated images through native reflection and reinforcement learning, achieving 64.9% repair success (up from 20.6%) and GenEval 0.84 (from 0.72) without external verifiers at inference.
BAGELGenEvalImage Generation
0 likes · 19 min read

Touch-R1 introduces the first tactile-anchored physical reasoning model using reinforcement learning to enable cross-sensor tactile inference, correcting visual misjudgments by prioritizing tactile evidence, and achieves state-of-the-art results on the new TouchReason-Bench benchmark across nine metrics including physical property identification, ordinal reasoning, and cross-sensor consistency.
Cross-Sensor ConsistencyEmbodied AINeurIPS 2026
0 likes · 10 min read

Researchers from Baidu, Renmin University, Tsinghua, and others introduce LoopVL, a recurrent vision-language model that reuses a 1B-parameter language backbone across 128 layer calls, achieving competitive performance with 260x less training data and exhibiting a 'Visual Aha Moment' where attention sharply refocuses on key image regions in the second loop.
Attention MechanismData EfficiencyLoop Transformer
0 likes · 14 min read

Researchers from Baidu, Renmin University, Tsinghua, and others introduce LoopVL, a recurrent vision-language model that reuses a 1B-parameter language backbone across 128 layer calls, achieving competitive performance with 260x less training data and exhibiting a 'Visual Aha Moment' where attention sharply refocuses across loops.
Attention MechanismLoop TransformerLoopVL
0 likes · 15 min read

UIUC postdoctoral researcher Chen Xusi will join the Chinese University of Hong Kong as a tenure-track assistant professor in fall 2027, focusing on self-evolving language agents, LLM reasoning and alignment, lifelong learning, multimodal understanding, AI for science, and sports analytics.
AI for ScienceLLM AlignmentLifelong Learning
0 likes · 6 min read

Princeton researchers introduce QUEEN, a 4B-parameter model that combines a strong chess engine with a language model via gated cross-attention bridges, trained through a four-stage QA curriculum and iterative search distillation to achieve 2697 Elo while generating natural-language move explanations.
Language ModelLeela Chess Zerochess AI
0 likes · 13 min read

This paper introduces CoLAS, a framework that explicitly models multimodal corroboration—non-cancelling support across heterogeneous financial modalities—to extract more reliable trading signals, achieving over 25% average annualized return improvement across six datasets including five stocks and Bitcoin.
corroboration miningcryptocurrency tradingdeep learning
0 likes · 18 min read

This survey reviews vision-language models for egocentric video, analyzing why standard VLMs fail on first-person footage and how hand-object interaction, spatiotemporal reasoning, graph-based representations, and semantic alignment bridge the gap toward embodied AI applications.
Embodied AIGraph Neural NetworksHand-Object Interaction
1 likes · 23 min read

Apple researchers propose Internalized Visual Thinking (IVT), a post-training framework that learns predictive world modeling during training but removes future-frame generation at inference, achieving 5x speedup over Visual CoT while matching or exceeding accuracy on Early-event and Next-event Prediction across Ego-Exo4D, Ego4D, and EPIC-KITCHENS-100.
Apple ResearchFlux-VAEInference Optimization
0 likes · 11 min read

The NTU PINE Lab introduces Facet-0, a multimodal robot foundation model that achieves 82% success in five real computer‑assembly tasks with 0.5 mm placement accuracy, reduces human intervention from 47% to 24%, and learns to recover from contact failures using a force‑synchronized dataset and reinforcement‑learning post‑training.
ManuFacet-1KNTURobotics
0 likes · 11 min read

The paper analyzes why Vision‑Language‑Action models fail when task instructions are paraphrased, demonstrates that language semantics remain partially encoded, and shows that the Grounded Semantic Re‑Binding (GSR) redesign of the language‑to‑action information flow improves instruction generalization by up to 40% across multiple VLA architectures.
GSRRoboticsVLA
0 likes · 17 min read

The article examines why vision‑language‑action models alone cannot fully understand physical environments, outlines the limitations of visual‑only embodied AI, and surveys recent algorithmic efforts—such as VTLA, N0‑VTLA, OmniVTLA—to embed tactile perception as a “touch‑neuron” for finer physical interaction.
Embodied AIWorld Modelsmultimodal learning
0 likes · 8 min read

The article presents a 30,000‑hour multimodal tactile‑visual dataset, unified representation models, and two downstream systems (VTLA and TWAM) that demonstrate how large‑scale touch data can dramatically improve robot manipulation success rates and establish tactile perception as a core scaling law for embodied intelligence.
Embodied AIRobot ManipulationScaling Law
0 likes · 9 min read

The paper introduces LIMSSR, an LLM‑driven sequence‑to‑score framework that addresses incomplete multimodal observations during training for fine‑grained action quality assessment, detailing its prompt‑guided modality completion, multi‑dimensional fusion, and mask‑aware dual‑path aggregation, and demonstrates state‑of‑the‑art results on FS1000, Fis‑V and RG benchmarks.
action quality assessmentincomplete observationslarge language model
0 likes · 10 min read

Weina AI’s Nature Communications paper demonstrates a multimodal deep‑learning model that predicts long‑term renal function decline after radical nephrectomy, achieves AUC 0.788‑0.873 across 15 centers, and showcases a closed‑loop inference‑data generation approach that moves AI from answering questions to autonomously creating high‑precision industry data.
AIAgentic AIclosed-loop feedback
0 likes · 12 min read

DeepStar’s VLIA model places intention understanding at the core of embodied AI, using a Vision‑Language‑Intention‑Action architecture, high‑bandwidth facial interfaces, and a data pipeline that scales intent‑supervised trajectories from short‑drama and game collaborations, aiming to turn household robots into true interactive partners.
Data ScalingEmbodied AIVLIA model
0 likes · 15 min read

The paper introduces the Temporal‑Aware Reasoning Optimization (TaRO) framework, which equips multimodal video large models with time‑aware reasoning via template‑based exploration, a temporal‑sensitivity reward, and progressive curriculum learning, achieving state‑of‑the‑art zero‑shot performance on several video temporal grounding benchmarks, including long‑video datasets.
TaROVideo Temporal GroundingZero-shot Evaluation
0 likes · 9 min read

The paper identifies routing drift and expert drift as the two main causes of forgetting in multimodal continual instruction tuning (MCIT) and proposes SAME, which combines spectral‑aware routing, curvature‑aware scaling, and adaptive expert activation to keep MoE models stable, efficient, and less forgetful across long task sequences.
Continual LearningICML 2026Same
0 likes · 19 min read

The paper introduces Deep Pre‑Alignment (DPA), a novel Vision‑Language Model architecture that inserts a perceiver VLM to pre‑align visual features with the LLM’s text space, reducing alignment cost, preserving language ability, and delivering consistent multimodal performance gains across multiple benchmarks with minimal inference overhead.
Deep Pre-AlignmentLLMPerceiver VLM
0 likes · 10 min read

The paper introduces the Spectral Disentanglement and Enhancement (SDE) framework, which uses singular value decomposition to separate strong semantic signals, weak auxiliary signals, and noise, applies curriculum‑based spectral enhancement, and jointly optimizes a dual‑domain contrastive loss, achieving markedly improved robustness and generalization on large‑scale multimodal benchmarks.
Contrastive Learningdual-domain lossmultimodal learning
0 likes · 14 min read

The article details how Beijing Humanoid’s Pelican‑Unify 1.0 model achieved top scores on WorldArena—including a 66.03 overall rating and 98.12% 3D accuracy—by unifying perception, reasoning, imagination and action in a single latent space, marking a milestone for model‑based end‑to‑end embodied intelligence.
Embodied AIPelican-UnifyRobotics
0 likes · 17 min read

Omni2Sound tackles the long‑standing “generalist” dilemma of unified audio generation by constructing a high‑quality V‑T‑A dataset (SoundAtlas), employing a three‑stage progressive training pipeline, and using a simple Diffusion Transformer backbone, ultimately achieving state‑of‑the‑art performance on T2A, V2A and VT2A tasks and strong robustness on off‑screen scenarios.
Omni2SoundTask Competitionaudio generation
0 likes · 16 min read

This article reviews eight Kuaishou‑authored papers accepted at ICLR 2026, summarizing their problem statements, novel methods such as front‑door causal attribution, visual table retrieval, denoising rerankers, difficulty‑adaptive reasoning, diffusion code infilling, generative ordinal regression, multimodal video retrieval, e‑commerce dialogue benchmarks, and a new LLM creativity evaluator, together with reported experimental gains.
ICLR 2026KuaishouOrdinal Regression
0 likes · 19 min read

LARYBench, the first large‑scale benchmark for embodied intelligence, quantifies implicit action representations across 1.2 million video clips, evaluates vision‑only and robot‑specific models, and reveals how general visual encoders can close the vision‑action modality gap.
Embodied AILARYBenchRobotics
0 likes · 12 min read

ControlAudio, a progressive diffusion model presented at ACL 2026, jointly models text, timing, and phoneme information to achieve precise event timing and intelligible speech in text-to-audio generation, backed by a large mixed real‑synthetic dataset and competitive experimental results.
ControlAudioProgressive DiffusionText-to-Audio
0 likes · 10 min read

The Point‑VLA method introduced by Qianxun AI’s Gaoyang team tackles the fundamental limits of language‑only instruction in vision‑language‑action models by adding visual grounding via bounding‑box cues, boosting real‑robot success rates from 32.4% to 92.5% across six challenging tasks.
Data AnnotationPoint-VLARobotics
0 likes · 13 min read

The newly released GigaWorld-Policy world‑action model replaces traditional video‑prediction‑heavy WAM designs with an action‑centered architecture, achieving a ten‑fold inference speedup, ten‑fold training efficiency gain, and a 30% increase in real‑robot task success rate while reducing memory usage compared with Motus and Cosmos‑Policy.
Action-Centered ArchitectureInference Optimizationmultimodal learning
0 likes · 8 min read

The paper reveals that standard shared‑parameter Transformers suffer severe gradient conflict when jointly processing low‑entropy text and high‑entropy visual tokens, and proposes Uni‑X—a two‑end‑separated, middle‑shared autoregressive model that isolates modality‑specific layers, reduces conflict, improves efficiency, and achieves strong results on image generation and editing benchmarks.
Autoregressive ModelICLR 2026Uni-X
0 likes · 8 min read

A new study suggests that large AI models could reduce their reliance on massive text corpora by early‑fusing multimodal data such as video and sensor streams, potentially slashing training costs, improving generalization, and prompting a shift toward more embodied notions of intelligence.
AI researchTraining Efficiencyembodied intelligence
0 likes · 6 min read

The article reviews the HORAI model, which introduces a frequency‑enhanced multimodal pretraining paradigm and the massive MM‑TS dataset, showing that integrating derived images, endogenous text, and real‑world news dramatically improves zero‑shot forecasting and anomaly detection across six domains.
HORAIPretrainingTime Series
0 likes · 23 min read

The International Conference on Learning Representations (ICLR) 2026 accepted multiple Xiaomi papers covering multimodal reasoning, reinforcement learning, GUI agents, autonomous driving, audio generation and benchmark design, each presenting novel frameworks, data‑centric training tricks and strong experimental results that advance the state of the art.
Autonomous DrivingICLR 2026Xiaomi
0 likes · 17 min read

The paper introduces H3M‑SSMoEs, a framework that integrates a multi‑context hypergraph for fine‑grained spatio‑temporal dynamics with a frozen Llama‑3.2‑1B LLM adapter, and a style‑structured expert mixture to jointly model stock relationships, multimodal semantics, and market regimes, achieving superior accuracy and investment returns on DJIA, NASDAQ‑100, and S&P‑100 benchmarks.
HypergraphLLMTime Series
0 likes · 14 min read

This article details how the Qwen‑Video‑8B model, built on Qwen3‑VL‑8B‑Instruct, is fine‑tuned with the LLaMA‑Factory framework using a curated city‑scenery dataset, addresses challenges of domain knowledge, temporal modeling and multimodal fusion, and demonstrates improved video captioning across baseline, English‑fine‑tuned and Chinese‑fine‑tuned versions.
AI fine-tuningLLaMA-FactoryLoRA
0 likes · 10 min read

The paper introduces DRFN, a dual‑relation fusion network that jointly models static and dynamic stock relationships using multimodal BERT and GRU encodings, achieving significantly lower RMSE and MAE than baseline models on both Chinese and US market datasets.
BERTGRUGraph Neural Network
0 likes · 11 min read

The article reports that CIKM 2025 in Seoul featured three Alibaba International Intelligent Technology papers on multimodal recommendation and dynamic reserve pricing, and that the team organized the AnalyticCup competition and a workshop to advance multilingual e‑commerce search using large language models.
CIKM 2025Dynamic Reserve PriceMultilingual Search
0 likes · 8 min read

The X‑VLA paper introduces a 0.9‑billion‑parameter, fully open‑source embodied model that uses a learnable soft‑prompt and divide‑and‑conquer encoding to handle heterogeneous robot vision inputs, achieving a record‑breaking 120‑minute autonomous clothing‑folding task while surpassing benchmarks across five simulation environments.
Embodied AIPretrainingRobotics
0 likes · 7 min read

This article summarizes six recent arXiv papers that explore how large language models, graph‑theoretic methods, generative frameworks, hypergraph multimodal architectures, GroupSHAP‑enhanced forecasting, and multi‑agent LLM workflows can improve financial signal extraction, portfolio optimization, and stock‑price prediction, providing empirical results on S&P 500 data.
LLMfinancial AImultimodal learning
0 likes · 13 min read

This digest presents seven recent arXiv papers that explore large‑language‑model‑driven portfolio scoring, hybrid ResNet‑RMT covariance denoising for crypto, LLM‑enhanced financial causal analysis, multilingual news alignment for stock returns, three‑step bubble prediction with news and macro data, multimodal volatility forecasting, and news‑aware reinforcement trading, each with reported performance gains.
Causal InferenceLLMfinancial AI
0 likes · 15 min read

It presents concise summaries of six recent arXiv papers on unsupervised domain adaptation, efficient forecasting, SHAP explanations, text‑reinforced multimodal forecasting, online prediction with feature adjustment, zero‑shot forecasting zoo, and a new anomaly‑detection metric, highlighting methods, datasets, and results.
SHAPTime Seriesanomaly detection
0 likes · 16 min read

Eight Kuashou research papers covering recommendation systems, multi‑task learning, multimodal large models, large language models, and combinatorial optimization have been accepted to the premier AI data‑mining conference KDD 2025, highlighting the company’s cutting‑edge innovations and their potential impact on the field.
AIdata miningmultimodal learning
0 likes · 18 min read

The VLDB‑2025 paper RCRank introduces a multimodal framework that collects slow‑query data, uses rule‑based and LLM analysis to identify root causes, quantifies their impact, and ranks them, achieving a 14% boost in optimization efficiency for cloud databases.
Root Cause AnalysisSlow Queriescloud databases
0 likes · 6 min read

This article details Xiaohongshu's multi‑stage recommendation pipeline—using massive multi‑modal pre‑training, long‑sequence modeling, real‑time context features, reinforcement learning and online deep learning—to precisely surface valuable content, address cold‑start challenges, and break information bubbles for billions of users.
Recommendation Systemlarge language modelmultimodal learning
0 likes · 16 min read

In a detailed interview, ByteDance AI specialist Cai Conghuai explains how large‑model techniques such as SFT, DPO and RAG address Douyin’s multimodal user‑experience challenges, improve signal detection, root‑cause analysis, and outline future AI‑agent breakthroughs for content platforms.
AI AlgorithmsRAGUser Experience
0 likes · 11 min read

This article explores DeepSeek’s cutting‑edge distillation technology, detailing its definition, underlying principles, innovative data‑model fusion, architecture choices, training strategies, performance gains over large language models, and the remaining challenges in knowledge transfer and multimodal data processing.
DeepSeekai-optimizationknowledge distillation
0 likes · 16 min read

This article examines the challenges of content safety, outlines the limitations of current task‑specific multimodal models, and proposes large‑model‑inspired training optimizations—including diversified data construction, automated annotation, parameter fine‑tuning, and multi‑task evaluation—to improve efficiency, accuracy, and scalability of multimodal AI systems.
Data Annotationai-optimizationcontent safety
0 likes · 26 min read

VideoCLIP‑XL, a new video CLIP model introduced by Alibaba Cloud AI Platform and Sun Yat‑sen University, enhances long‑text description comprehension through a large‑scale VILD dataset, a text‑similarity guided principal component matching method, and novel DDR and HDR ranking tasks, achieving superior performance on multiple video‑text benchmarks.
DatasetLong DescriptionVideo CLIP
0 likes · 13 min read

This article details Xiaohongshu's technical approaches to decentralized content distribution, covering business background, core challenges, high‑frequency recommendation pipelines, link‑level analysis, sideinfo decoupling, graph‑model integration, multimodal signal fusion, explicit interest exploration, interest protection, and future research directions.
decentralized-distributiongraph modelsinterest-exploration
0 likes · 24 min read

This article details how NetEase Cloud Music leverages multimodal content understanding—using audio models like MusicCLIP and Audio MAE and image‑text fusion via FLAVA—to improve recommendation performance for new content and new users, covering system architecture, cold‑start solutions, and future AI‑driven directions.
AI modelsRecommendation Systemaudio representation
0 likes · 15 min read

The paper presents a multimodal video scene classification system that leverages CLIP‑generated pseudo‑labels and a fine‑tuned image encoder to automatically identify nature, animation/game, and document scenes, enabling more effective adaptive transcoding, intelligent restoration, and quality assessment for user‑generated content on platforms such as Bilibili.
Bilibili multimediaCLIPComputer Vision
0 likes · 17 min read


AntTechAug 16, 2024 · Artificial Intelligence
The paper introduces PC², a novel framework that combines pseudo‑classification and pseudo‑captioning to mitigate noisy correspondence in cross‑modal retrieval, presents a large‑scale web‑page/image‑meta‑description dataset called Noise of Web (NoW), and demonstrates significant performance gains on multiple benchmark datasets including Flickr30K, MS‑COCO, and the newly released NoW.
PC2cross-modal retrievalmultimodal learning
0 likes · 16 min read

Alibaba’s advertising team introduces semantic‑aware contrastive learning to pre‑train multimodal image‑text embeddings, integrates them via SimTier and MAKE into ID‑based CTR models, achieving up to 6.9% lift in Taobao display ad click‑through rates and improving long‑tail item performance.
CTR predictionContrastive Learninge-commerce
0 likes · 21 min read

The paper presented at AAAI introduces the EERCF method, a coarse‑to‑fine visual representation and two‑stage recall‑then‑rerank strategy that dramatically reduces cross‑modal matching FLOPs while preserving state‑of‑the‑art retrieval performance on multiple video benchmarks.
AIEfficiencycoarse-to-fine representation
0 likes · 8 min read

This curated reading list compiles essential papers on AI agents, task planning, hallucination mitigation, multimodal models, image/video generation, foundational LLM research, open‑source large models, fine‑tuning techniques, and performance optimization, providing a comprehensive roadmap for anyone aiming to master modern generative AI.
AI agentsmultimodal learningperformance optimization
0 likes · 23 min read

The article examines NetEase Cloud Music’s video and image technology stack—covering a four‑module architecture, algorithms for content understanding, intelligent production, moderation, and interactive effects—and explains how these systems enhance user experience, streamline backend processing, and position the platform for future AIGC‑driven innovations.
AI Algorithmscontent enhancementimage recognition
0 likes · 11 min read

This article presents NetEase Cloud Music's multimodal cold-start recommendation approach, detailing the problem's significance, feature extraction using CLIP, I2I2U indirect modeling, U2I DSSM direct modeling with contrastive learning and interest‑boundary mechanisms, deployment pipeline, evaluation results, and future optimization directions.
Contrastive Learningcold startmultimodal learning
0 likes · 14 min read

At CVPR 2023 the Xiaohongshu team presented OvarNet, a unified one‑stage Faster‑RCNN model built on CLIP that uses prompt learning and knowledge distillation to jointly detect objects and recognize open‑vocabulary attributes, achieving state‑of‑the‑art results on VAW, MS‑COCO, LSA and OVAD datasets.
Computer VisionObject Detectionattribute recognition
0 likes · 12 min read

The article presents OPPO's latest research on virtual human audio‑lip and RGB driving, multimodal learning breakthroughs such as CETNETs and cross‑modal matching, and a reflective discussion on the challenges and future directions of general artificial intelligence, highlighting the interconnections among these three domains.
AI EngineeringGeneral AINeRF
0 likes · 9 min read

This article explains how a multimodal deep learning model combines static and temporal game data to predict frame rates, helping identify performance bottlenecks and improve client smoothness through feature fusion, data pipelines, and real‑time inference in modern games.
AIdeep learningfeature engineering
0 likes · 7 min read

This article reviews Huawei Noah's Ark Lab's work on modern information‑flow recommendation, covering the evolution from collaborative filtering to deep learning, the application of BERT‑based pre‑training for news ranking, multimodal user‑interface modeling, practical deployment challenges, and future research directions.
AIBERTHuawei
0 likes · 19 min read

To tackle the massive, multimodal tagging challenge of short‑video platforms—characterized by a huge long‑tail tag set, sparse annotations, and uneven modality contributions—the authors propose a two‑stage recall‑ranking system that first retrieves candidates via text, visual, audio and classification cues, then refines them with contrastive learning and extensive hard‑negative sampling, achieving 0.884 tag accuracy in a real‑world news video recommender.
Embeddingmultimodal learningrecall ranking
0 likes · 12 min read

This article describes the end‑to‑end multimodal modeling pipeline—covering text, image, and video understanding, model evolution from shallow to deep networks, key‑frame extraction, fine‑tuning, and multimodal fusion—used in Tencent's game ad exchange platform, along with practical deployment challenges and solutions.
AdvertisingCNNTransformer
0 likes · 22 min read

This article surveys PaddleVideo, Baidu's open‑source video AI toolkit, detailing its industry‑focused models for sports action recognition, multimodal tagging, intelligent production, interactive segmentation, drone detection, and medical imaging, while providing performance metrics and GitHub resources for each solution.
Computer VisionPaddleVideoVideo AI
0 likes · 14 min read

This article presents the multimodal learning demands of QQ Music, introduces the MMatch series of multimodal matching technologies—including image‑text matching, music similarity, AI tagging, and video scoring—and details their practical applications in business scenarios such as merchant public‑play, search, recommendation, and future product ideas.
Tencent Musicartificial intelligenceembedding matching
0 likes · 25 min read


Amap TechNov 4, 2021 · Artificial Intelligence
To efficiently filter unchanged POI signboards, the authors propose a multimodal image‑retrieval system that combines enhanced global and local visual features with BERT‑encoded OCR text, using metric learning and alignment techniques to achieve over 95 % accuracy while handling occlusion, viewpoint variation, and subtle text changes.
Computer VisionPOIdeep learning
0 likes · 17 min read

This article explains the challenges of POI signboard image retrieval, describes a multimodal deep‑learning solution that combines visual and OCR‑based text features, details data generation, model architecture, loss functions, and presents impressive accuracy improvements and future research directions.
POI mappingdeep learningimage retrieval
0 likes · 17 min read

Meituan’s To‑Store Integrated data team built an end‑to‑end supply‑standardization pipeline for the rapidly growing script‑murder market by extending the GENE knowledge graph to mine merchant supply, construct a unified script library through rule‑based, semantic, and multimodal clustering, and link products and user‑generated content to standardized scripts, enabling a dedicated category, personalized recommendations, filter tags, and improved ranking.
BERTRecommendation Systementity linking
0 likes · 23 min read

This paper proposes HiT, a hierarchical transformer model with momentum contrast for video-text retrieval, addressing limitations in existing multimodal learning methods by introducing hierarchical cross-modal contrast matching and momentum cross-modal contrast to improve retrieval performance.
Contrastive LearningHCMMCC
0 likes · 9 min read

This article presents a comprehensive overview of AI‑driven intelligent grading technologies, covering background, typical educational challenges, multimodal NLP solutions for essay, spelling and grammar correction, adaptive learning, and related research, illustrating how deep learning and multimodal models improve automated assessment across K‑12 scenarios.
AIEducation TechnologyEssay Scoring
0 likes · 24 min read

The article outlines the Tencent Advertising Algorithm Competition’s video ad challenge, details the paper submission guidelines, and shares a participant’s step‑by‑step technical approach—including baseline experiments, model re‑implementation with Paddle, multimodal feature extraction, optimizer choices, and future improvement directions—providing practical AI insights for multimedia video classification.
Tencent competitiondeep learningfeature extraction
0 likes · 7 min read

This article surveys recent multimodal AI research, covering video scene‑aware dialog with a GPT‑2 based unified pre‑training framework, dual‑channel multi‑hop reasoning for visual dialog, capsule‑network‑enhanced multimodal machine translation, and graph‑neural‑network‑driven multimodal translation, highlighting experimental results and future directions.
Graph Neural Networkmachine translationmultimodal AI
0 likes · 12 min read


Amap TechJan 15, 2021 · Artificial Intelligence
To tackle the AMAP‑TECH competition’s dynamic road‑condition classification from scarce, imbalanced vehicle‑video frames, the team combined YOLOv5 object detection, ResNeXt101‑based semantic embeddings, and engineered temporal detection statistics, feeding the fused features into a five‑fold LightGBM model that achieved top weighted‑F1 performance.
Computer VisionLightGBMResNeXt
0 likes · 10 min read

Meituan’s Search Ads team placed third in the KDD Cup 2020 Multimodalities Recall track by tackling training‑test distribution mismatch with diversified negative sampling and distillation learning, and improving text‑image matching via gated fully‑connected layers, bidirectional attention, and diversified fusion, then ensembling neural and tree models for strong NDCG gains later applied to their ad creative‑selection system.
KDD CupText‑Image Matchingdistillation
0 likes · 19 min read

At the SIGIR 2020 Meituan workshop, researchers unveiled MT‑BERT’s large‑scale pre‑training and compression techniques, a KDD Cup winning solution that tackles bias with graph‑ and multimodal learning for search advertising, and a massive food‑delivery knowledge graph powering personalized recommendations, all demonstrating significant real‑world performance gains.
model compressionmultimodal learningpretrained language models
0 likes · 18 min read

This article describes iQIYI's short‑video content tagging system, outlining the challenges of extracting type and abstract tags from multimodal data, detailing the evolution from text‑only models to image‑fusion, BERT‑enhanced, and video‑frame models, and discussing their applications and future directions.
BERTTransformercontent tagging
0 likes · 11 min read

This article examines the challenges of short‑video content tagging and describes iQIYI's multi‑stage evolution from simple text‑only models to sophisticated multimodal architectures that fuse cover images, BERT embeddings, and video frames to improve tag generation accuracy.
BERTTransformercontent tagging
0 likes · 12 min read

iQIYI’s short‑video tagging system tackles multimodal fusion, open‑set and abstract tags by evolving from a text‑only model through cover‑image, BERT‑vector, and video‑frame fusion architectures, enabling automated labeling, personalized recommendation, and semantic search while planning to add OCR, audio, and knowledge‑graph enhancements.
BERTTransformercontent tagging
0 likes · 13 min read

This article reviews a AAAI 2020 paper that introduces a language‑attention based Visual Question Answering model which decomposes questions into type, object, and concept expressions to mitigate language bias, explains its modular architecture, and demonstrates superior performance on VQA‑CP v2 through extensive experiments and ablations.
Attention MechanismVQA-CPVisual Question Answering
0 likes · 14 min read

This article presents a comprehensive overview of UC's information‑flow video tag recognition technology, detailing tag usage scenarios, the end‑to‑end system architecture, multi‑modal feature extraction, advanced deep‑learning models such as NextVlad, behavior and person tagging methods, and future research directions.
Computer Visiondeep learningmultimodal learning
0 likes · 14 min read

This article presents a Transformer‑based framework that simultaneously generates visually compatible outfits and personalizes recommendations by leveraging multimodal item embeddings and user behavior, achieving significant gains in compatibility prediction, fill‑in‑the‑blank accuracy, and click‑through rate on Alibaba's iFashion platform.
Transformerdeep learninge‑commerce
0 likes · 15 min read

The authors introduce a feature‑fusion pipeline and a Residual Dense Net that leverages multi‑frame face embeddings to identify persons in iQIYI‑VID‑2019 videos, achieving 0.9035 mAP (second place) with only ≈0.5 GFLOPs and processing the full test set in minutes.
Feature FusioniQIYI-VID-2019multimodal learning
0 likes · 11 min read

Watchdog team won TOP1 in iQIYI & ACMMM2019 multimodal video person recognition challenge using pre‑extracted multimodal features, a 2048‑dim classifier with BCE loss, re‑ranking, DALI‑accelerated re‑detection, fine‑tuned InsightFace, and multi‑model ensembling achieving ~91% test accuracy.
Feature FusionRe-rankingmodel ensemble
0 likes · 12 min read

This article explores multimodal representation learning by introducing a Bilinear Residual Layer that automatically fuses image and text features, demonstrates its superiority over traditional concatenation and FiLM methods on text‑guided image editing and fashion synthesis tasks, and reports state‑of‑the‑art results on several benchmark datasets.
GANbilinear residual layercross-modal attention
0 likes · 17 min read


JD TechJan 30, 2019 · Artificial Intelligence
At AAAI 2019 in Hawaii, JD AI Research Institute had eight papers accepted covering machine learning, natural language processing, computer vision, and multimodal AI, highlighting innovations such as AutoZOOM black‑box attacks, SACN for knowledge base completion, and temporally aware video captioning models.
Computer Visionartificial intelligencemultimodal learning
0 likes · 11 min read

The paper presents an integrated multimodal quality assessment system for short‑video platforms that evaluates cover images, video content, and accompanying text using deep‑learning and handcrafted features—combining ResNet‑50, NetVLAD, TSN, VGGish, and XGBoost—to improve user experience, recommendation accuracy, and operational efficiency, with plans for optimization and modular deployment.
image analysismultimodal learningtext classification
0 likes · 11 min read

This article reviews an ICCV paper introducing a hierarchical multimodal LSTM that jointly embeds images, phrases, and whole sentences, enabling detailed image descriptions and superior performance on Flickr30K, MS‑COCO, and region‑phrase datasets compared to previous methods.
Computer VisionImage Captioninghierarchical LSTM
0 likes · 8 min read

The Alibaba iDST team clinched first place in the ACM MM2017 LSVC competition by leveraging Alibaba Cloud’s ODPS to extract eight multimodal features, achieving a 0.8485 mAP on the validation set, and demonstrating the critical role of rich modality fusion in large‑scale video classification.
AlibabaODPSlarge-scale AI
0 likes · 5 min read

In an exclusive interview, Alibaba AI Lab's distinguished scientist Dr. Wang Gang discusses the lab's research on multimodal AI, edge computing, AI hardware, bio‑inspired cognition, quantum‑deep‑learning integration, and the challenges of moving from recognition to true understanding, while also outlining Alibaba's AI talent recruitment plans.
AI researchAI talent recruitmentComputer Vision
0 likes · 25 min read
