Can Ontology‑Aware KG‑RAG Double Table QA Performance on Industrial Standards?

Background

Industrial standards in sectors such as shipbuilding, offshore, and energy (e.g., ASTM, API, ISO) are typically massive, deeply nested documents where a single clause may contain multiple conditions, exceptions, numeric values, and units. Tables often embed row and column conditions, unit conversions, and footnote exceptions, making direct retrieval challenging.

Problems with Traditional RAG

Information overload : Splitting a 60‑page document into 512‑token chunks drowns the LLM in noise, hiding critical clauses behind references.

Table failure : Row‑column relationships are fragmented, preventing alignment of condition‑conclusion pairs.

Numeric hallucination : Units and boundary values are freely invented by the model.

Proposed Ontology‑Aware KG‑RAG Pipeline

The authors introduce a three‑stage pipeline that encodes both document structure and business rules into a knowledge graph.

Stage ① Hierarchical Ontology Modeling : Map chapter‑section‑table‑footnote hierarchy into a tree. Example mapping: Section → Subsection → Table → Footnote. Output: document skeleton graph.

Stage ② Atomic Proposition Modeling : Use an LLM to recursively decompose conditional sentences (e.g., “If A and B or C, then D”) into atomic propositions, then into triples. Output: condition‑conclusion triples.

Stage ③ KG Refinement : Perform synonym clustering and pruning to compress ~50 k redundant triples into ~5 k high‑quality edges, yielding an inferable knowledge graph.

Key Technical Details

Table→Proposition : Treat each cell as a “case” node. Row/column headers become has_condition_AND/OR edges, cell values become has_consequence edges.

Numeric Normalization : Automatically convert all imperial units to SI to avoid duplicate entities.

Synonym Dictionary : Apply Sentence‑BERT + HDBSCAN clustering to map synonyms such as “tensile strength/抗拉强度/TS” to a single node.

Two‑Stage Retrieval :

Ontology‑Level: Locate the most relevant “section” node first, then expand 1–2 hops.

Global-Level: Perform semantic vector search over the entire graph to retrieve cross‑section, multi‑hop evidence.

Experimental Evaluation

The authors built the IndusSpec‑QA benchmark from three ASTM/API standards (total 95 pages) containing 1 548 manually verified questions covering tables, rules, multi‑hop reasoning, and toxic‑clause detection.

Compared models include DeepSeek‑v3.1, Gemini‑2.0‑flash, Dense+Gemini‑2.0, and BM25+Qwen1.5‑14B.

Results:

Ontology‑aware KG‑RAG achieved an average F1 of 0.454 , a 93.7 % improvement over the strongest text‑only RAG baseline (0.277) and a 49 % gain over traditional KG‑RAG (0.304).

Table‑type questions saw the largest boost, effectively doubling F1 because condition‑conclusion pairs were explicitly represented as triples.

Toxic‑clause detection recall increased from 0.88 to 0.91, potentially saving millions of dollars by catching fatal clauses.

Robustness: Performance remained stable on a 63‑page A6/A6M document, demonstrating that structural priors outperform brute‑force chunking for long texts.

Ablation studies showed that using the ontology alone outperformed using the KG alone, confirming that converting section numbers into graph nodes provides the strongest signal.

Additional Highlights

Toxic clause detection : Recall improvement from 0.88 to 0.91 reduces risk of missing critical safety clauses.

Long‑document robustness : Stable results on 63‑page standards show the advantage of structural priors over naive chunking.

Module ablation : Using only the ontology yields higher performance than using only the KG, emphasizing the importance of mapping hierarchical identifiers to graph nodes.