LiRA: A Linguistic Robust Anchoring Framework for Low-Resource Cross-Lingual LLM Retrieval

Background

Large language models (LLMs) have reached near‑saturation performance on high‑resource languages such as English and Chinese, but their capabilities on low‑resource languages (e.g., Thai, Urdu, Bengali) remain far behind because of limited training data, translation noise, and unstable cross‑lingual alignment. Improving cross‑lingual representations for retrieval and reasoning is essential for accurate search intent detection and ad recall in emerging markets.

LiRA Framework Overview

LiRA (Linguistic Robust Anchoring) decomposes cross‑lingual adaptation into two complementary stages.

Arca (Anchored Representation Composition Architecture) aligns low‑resource language inputs to the English semantic space via a dual‑path mechanism: an anchor path that maps directly to English embeddings and a translation path that processes the machine‑translated English version.

LaSR (Language‑Coupled Semantic Reasoner) builds on Arca’s multilingual embeddings, adding a lightweight Transformer head that fuses the two paths and jointly optimizes retrieval, similarity, and reasoning objectives.

The two paths are coordinated by a critic‑actor interaction: a Translation Critic evaluates candidate translations on semantic fidelity, emotional consistency, and pragmatic tone; an Embedding Critic enforces geometric consistency in the shared embedding space; and an Actor aggregates the signals to produce the final anchored representation.

Theoretical Foundations

LiRA formalizes semantic anchoring with the following definitions and assumptions:

Assumption 1 (Semantic Anchoring) : the mismatch between anchor and translation paths is bounded.

Assumption 2 (Translation Fidelity) : the translation operator preserves semantic distribution under a KL‑divergence bound.

Definition 1 (RKHS Representation) : English sentence embeddings are modeled as kernel mean embeddings in a reproducing kernel Hilbert space.

Definition 2 (Local Lipschitz Continuity) : encoders possess a bounded Lipschitz constant on finite discrete domains.

Under these assumptions, the Representation Deviation Theorem proves that the anchored output deviates from the optimal English representation by a quantifiable error term, and the Downstream Stability Corollary guarantees that, when the local Lipschitz constant is small, both representation bias and downstream performance gap converge to zero.

Arca Design Details

The critic‑actor loop reduces two primary error sources:

Embedding Critic minimizes the anchoring error by regressing the anchor‑path vector toward the translation‑path vector, thereby preserving geometric stability.

Translation Critic scores each candidate translation on three dimensions—semantic fidelity, emotional consistency, and pragmatic tone—using a lightweight LLM evaluator. These scores serve as KL‑divergence proxies and are maximized during policy optimization.

The combined loss contracts the anchoring radius and aligns the multilingual representation with the English semantic manifold.

LaSR Design Details

LaSR introduces two FIFO queues to address data sparsity in low‑resource settings:

CorrQueue caches (prediction, gold) pairs from ranking datasets, enabling a composite relevance loss over current and historical batches.

DocQueue stores recent (doc‑id, language, embedding) tuples, supporting in‑language hard‑negative mining and a differentiable soft‑nDCG@k objective.

The training pipeline alternates between updating the queues and computing a weighted loss L = λ_r * L_{corrQ} + λ_t * L_{retr}. Temperature scheduling (high temperature early, low later) and gradient clipping (threshold 1.0) stabilize optimization. The full algorithm is shown below:

Input: query batch Q, document batch D, queue capacity K
Output: updated parameters θ
1: forward Q and D through shared encoder and LLM transformer
2: obtain query embeddings {ẑ_q} and document embeddings {ẑ_d}
3: for each (q,d) pair do
4:   compute similarity s(q,d) = ẑ_q^T ẑ_d
5: end for
6: update CorrQueue with (pred,gold) pairs, evict oldest
7: update DocQueue with (doc_id,lang,ẑ_d), evict oldest
8: compute composite loss L = λ_r * L_corrQ + λ_t * L_retr
9: back‑propagate and update θ
10: return θ

Experimental Setup

We evaluate LiRA on three task families:

Retrieval (nDCG@10) using Qwen3‑Embedding‑8B.

Sentence similarity (Pearson) on STS22.

Complex reasoning (accuracy) on MGSM (math) and X‑CSQA (reading comprehension) using Qwen3‑4B.

Datasets include public benchmarks (BelebeleRetrieval, MLQARetrieval, STS22) and two newly released e‑commerce retrieval sets: LazRetrieval (10 k samples per language) and LazRetrieval‑mega (1 M samples per language) covering Vietnamese, Thai, Indonesian, Malay, Urdu, Bengali, and Tagalog. All models are fine‑tuned with identical hyper‑parameters; each experiment is repeated ten times and averaged.

Main Results

On LazRetrieval, LiRA raises average nDCG@10 from 68.05 % to 77.71 % (+9.66 %). Gains are especially pronounced for extremely low‑resource languages (e.g., Urdu +18.16 %, Bengali +15.71 %). On public benchmarks, LiRA consistently outperforms the strong Qwen3‑E‑8B baseline: MLQARetrieval 82.01 % vs 81.13 % (+0.88 %), BelebeleRetrieval 87.03 % vs 85.94 % (+1.09 %), STS22 Pearson 75.00 % vs 71.64 % (+3.36 %). In MGSM and X‑CSQA, LiRA improves average accuracy from 62.9 % to 65.5 % (+2.6 %) and shows the largest lifts on low‑resource or typologically distant languages.

Ablation Study

Removing the Translation Critic or Embedding Critic causes the largest drops, especially on Pearson (‑2.81 % and ‑13.22 % respectively). Excluding either representation path also degrades performance, confirming their complementary nature. Disabling the FIFO queues reduces nDCG@10 from 77.71 % to 64.29 % (‑13.42 %), highlighting the importance of historical sample reuse for stable gradient estimation in low‑resource regimes.

Conclusion and Future Work

LiRA demonstrates that a theoretically grounded, dual‑path anchoring strategy combined with lightweight semantic reasoning yields robust cross‑lingual performance, especially for low‑resource languages. Future directions include:

Incorporating richer linguistic priors (language families, typological features) to build hierarchical anchoring.

Extending LiRA to multimodal alignment by using images as language‑agnostic anchors.

Designing incremental learning mechanisms for continual adaptation to new languages and evolving vocabularies.

Integrating advanced LLM reasoning techniques (chain‑of‑thought, program‑aided inference) to further boost multilingual reasoning capabilities.