From Black‑Box Guessing to Quantitative Deconstruction: Unveiling the Mystery Inside Large Language Models

EMNLP (Conference on Empirical Methods in Natural Language Processing) is a top‑tier NLP conference organized by ACL's SIGDAT. The 2025 edition was held in Suzhou, China, receiving over 4,300 submissions and attracting thousands of researchers.

Paper Presentation

The first author presented the BUPT NIRC team's work titled Unveiling Internal Reasoning Modes in LLMs . The study focuses on multi‑hop questions such as “What is the capital of the country where the Eiffel Tower is located?” where LLMs often produce the correct answer but follow different internal paths.

Latent Reasoning : the model truly understands the logic, first identifying the intermediate entity (France) and then deriving the answer (Paris).

Factual Shortcuts : the model skips logical chaining and directly outputs the answer by exploiting strong co‑occurrence patterns (e.g., “Eiffel Tower” ↔ “Paris”) in the training data.

The goal is to distinguish these two modes and locate their essential differences within the model’s layer hierarchy.

ARR Metric

The authors propose the ARR (Attribute Retrieval Rate) metric to quantify how quickly a model extracts attribute information during reasoning.

Design: Using the Logit Lens technique, the contribution of each layer to the final prediction is observed.

Finding: Experiments show that ARR can automatically classify the two reasoning states with 90% accuracy, eliminating the need for manual labeling and greatly improving the efficiency of interpretability research.

Model Dissection

Two analysis techniques—Attention Knockout and Logit Lens—were applied:

Latent reasoning path: middle layers significantly encode “bridge entity” information, indicating that the model performs a genuine logical jump.

Shortcut mode: early layers already lock onto the final answer, behaving more like memory retrieval than logical inference.

Conference Experience

On the morning of November 15, the author set up a poster. Initially nervous about English fluency, they soon found that explaining the ARR metric and how LLMs differentiate reasoning from shortcuts sparked lively discussions, breaking language barriers.

Interactions continued through the poster session, informal gatherings, and a night‑time social event featuring a Chinese lion dance, where researchers exchanged ideas on scaling laws and RAG robustness.

Overall, the presentation not only introduced a novel quantitative tool for probing LLM internals but also highlighted the collaborative and cultural richness of the EMNLP community.