Stop Fragmenting Long Texts: HiLight Lets AI Highlight Key Points Directly

Large language models often suffer from the "Lost in the Middle" problem, where attention to information appearing in the middle of the input drops sharply. Existing solutions fall into two categories: hard selection (retrieving or cutting relevant fragments, risking loss of essential context) and soft selection (summarizing or compressing the input, which can introduce distortion).

HiLight proposes a new "input-side intervention": inserting a small number of high‑light tags directly into the original text to steer the model’s attention. The tags, such as <start_important> and <end_important>, mark important spans without removing any surrounding context.

Because many LLMs are accessed via paid APIs and their weights are not open, fine‑tuning is often impractical. HiLight therefore freezes the inference‑time solver LLM and trains a lightweight "Emphasis Actor" that reads the full context, assigns an importance score to each token, and inserts the highlight tags before passing the text to the frozen solver.

The training loop uses only the solver’s task reward (e.g., HR@10, EM, F1) as feedback, treating highlight selection as a reinforcement‑learning problem. A highlight‑budget mechanism limits the proportion of tokens that can be marked and merges scattered token‑level selections into coherent spans, preventing the actor from trivially highlighting everything.

Experiments on four downstream tasks—Amazon‑Beauty (sequential recommendation), HotpotQA (multi‑hop QA), SQuAD 2.0 (reading comprehension), and PubMedQA (biomedical classification)—compare HiLight against several prompt‑optimization baselines (PRL, BFRS, OPRO, DSPy/MIPROv2, APE). The largest gain appears on the recommendation task, while the other tasks show consistent, albeit modest, improvements.

Ablation studies contrast feeding the solver only the highlighted fragments (cutting) versus HiLight’s full‑text with tags. Cutting works for Amazon‑Beauty but harms HotpotQA because multi‑hop reasoning requires preserved contextual connections; HiLight retains the full context while still emphasizing key evidence.

The actor’s learned highlighting strategy transfers across solvers. An actor trained with Qwen3‑14B as the solver was applied unchanged to five unseen solvers, outperforming each solver’s own self‑highlighting. The advantage stems from the actor being explicitly trained with task rewards to recognize evidence that truly boosts downstream metrics.

Although no token‑level human annotations are used during training, the actor’s highlighted spans align closely with human‑annotated supporting facts in HotpotQA, achieving up to 0.78 F1. Scaling the actor from 0.6 B to 8 B parameters raises precision, recall, and F1 monotonically, with precision reaching 0.84, indicating that most highlighted tokens correspond to human‑identified key evidence.

Deployment costs are low: the solver’s token overhead increases by less than 1.01×, actor inference latency is negligible (≈0.05 s for a 0.6 B model, ≈0.23 s for a 4 B model, compared to 8–18 s for the solver), and training requires only about 12 K solver calls versus 120 K for PRL and 60 K for APE.

In a concrete Amazon‑Beauty case, the actor highlighted two crucial pieces of information, moving the target product’s rank from 14 to 5, demonstrating a clear, interpretable improvement.

Overall, HiLight offers several benefits: up to 27 % performance gains on recommendation tasks, no need to modify the frozen solver, direct explainability through visible tags, cross‑model transferability, and minimal additional latency and training cost. As API‑based LLM usage grows, HiLight provides a practical way to boost performance without altering the underlying solver.