Enabling Unseen Language QA Without Training LLMs: XBridge’s Plug‑in Multilingual Extension

Overview

XBridge is a new multilingual extension paradigm proposed by the NLP team at the Institute of Computing Technology, Chinese Academy of Sciences. It integrates the general knowledge‑processing ability of an English‑centered large language model (LLM) with the multilingual understanding and generation capability of existing neural machine translation (NMT) models, achieving a complementary multi‑language system without any LLM fine‑tuning.

Model Architecture

The system follows an encoder‑LLM‑decoder three‑stage pipeline. An NMT encoder maps multilingual input into a shared semantic space, the LLM performs English‑centric knowledge reasoning on that representation, and an NMT decoder generates the target‑language output. Lightweight MLP mapping layers connect the three modules.

Optimal‑Transport Alignment

Because the encoder, LLM, and decoder operate in heterogeneous token‑level spaces, a simple MLP mapping cannot guarantee semantic consistency. XBridge therefore introduces an optimal‑transport (OT) alignment objective that learns soft token‑level matches, enabling fine‑grained semantic alignment across different tokenizations and sequence lengths and ensuring stable, high‑quality multilingual generation.

Three‑Stage Training Strategy

Cross‑model alignment stage: learns the basic semantic mapping among encoder, LLM, and decoder.

Encoder adaptation stage: teaches the LLM to exploit encoder representations for downstream tasks.

Decoder adaptation stage: further improves multilingual generation quality.

The staged design avoids conflicts between different optimization goals and gradually builds a stable cross‑model mapping that can be adapted to downstream tasks.

Experimental Results

Multilingual capability transfer : On the FLORES‑101 translation benchmark, XBridge markedly improves LLM performance on low‑resource or unseen languages such as Bengali and Swahili, reaching or surpassing the quality of dedicated NMT models.

Downstream tasks : On multilingual math reasoning (MGSM) and summarization (XL‑Sum), XBridge yields significant gains for low‑resource languages, narrowing the performance gap with high‑resource languages while preserving or improving high‑resource language results—all without training the LLM.

Language‑agnostic cross‑model mapping : The system generalizes well to languages never seen during training, indicating that XBridge learns a language‑independent mapping; OT alignment contributes to this generalization.

Controllable generation and lossless language switching : By feeding a language label to the decoder, XBridge can produce output in any target language, enabling arbitrary language‑to‑language generation with seamless, lossless switching.

Demo and Resources

A demo trained on 50 languages demonstrates multilingual question answering and language‑switching capabilities. Resources are publicly available:

Paper: https://arxiv.org/abs/2603.17512

Code: https://github.com/ictnlp/XBridge

Model hub: https://huggingface.co/collections/ICTNLP/xbridge

Conclusion and Outlook

By offloading multilingual understanding and generation to an external NMT model, XBridge achieves high‑quality support for low‑resource and unseen languages without any LLM training. This approach suggests a new, low‑cost pathway for extending LLM multilingual abilities, potentially reducing reliance on massive multilingual pre‑training data.