How Syntax‑Sensitive Entity Representations Boost Neural Relation Extraction

Abstract

Syntax‑sensitive entity representations are applied to neural relation extraction. A major bottleneck for large‑scale relation extraction is the acquisition of annotated corpora. Recent neural models map sentences to low‑dimensional vectors. This work innovates by incorporating syntactic information into entity representation models, first using a Tree‑GRU to embed dependency trees of entity contexts at the sentence level, and then applying inter‑sentence and intra‑sentence attention to obtain representations of sentence collections containing target entities.

Research Background and Motivation

The large‑scale deployment of relation extraction is hindered by the difficulty of obtaining labeled data. Distant supervision aligns knowledge bases with unstructured text to automatically generate training data, reducing reliance on manual annotation and improving cross‑domain adaptability. However, distant supervision often aligns only entity names, ignoring richer semantic variations of entities across different relations, leading to noisy labels. Therefore, richer entity representations are essential.

Related Work

Related work can be divided into early distant‑supervision methods and recent neural‑network‑based approaches. Mintz et al. (2009) first proposed distant supervision for building labeled corpora, but the resulting data contain substantial noise. Subsequent works (Riedel et al., 2010; Hoffmann et al., 2011; Surdeanu et al., 2012) modeled relation extraction as multi‑instance learning and incorporated shortest dependency paths as syntactic features, yet they relied heavily on handcrafted feature templates. More recent neural methods include Socher et al. (2012) using recurrent networks, Zeng et al. (2014) employing end‑to‑end convolutional networks, and Lin et al. (2016) applying attention mechanisms to select informative instances. Most of these neural models use word‑level representations for sentence encoding, while syntax‑based representations such as shortest dependency paths (Miwa & Bansal, 2016; Cai et al., 2016) have also attracted attention.

Main Method

The proposed method first leverages dependency parse trees and a Tree‑GRU model to generate sentence‑level representations of entities, capturing long‑distance information beyond the entity surface form. The Tree‑GRU aggregates information from child nodes to produce a semantic vector for each entity.

Next, two attention mechanisms are employed: a child‑node attention (ATTCE) to mitigate syntactic errors, and a sentence‑level entity attention (ATTEE) to aggregate representations of multiple sentences containing the target entity, thereby reducing the impact of noisy annotations.

Experimental Results

Experiments were conducted on the NYT corpus. Two strategies, SEE‑CAT and SEE‑TRAINS, combine three vector representations (sentence vector, two entity vectors). The results show that the proposed model outperforms existing distant‑supervision relation extraction models on the same dataset.

Conclusion

The experimental results demonstrate that richer semantic representations of named entities can effectively enhance the performance of relation extraction tasks.