Syntax Sensitive Entity Representation for Neural Network Relation Extraction

Summary

Syntax-sensitive entity representation for neural network relation extraction. A major bottleneck in the large-scale application of relation extraction tasks is the acquisition of corpus. In recent years, neural network-based relation extraction models represent sentences in a low-dimensional space. The innovation of this paper is to add syntactic information to the representation model of entities. First, based on Tree-GRU, the dependency tree of the entity context is put into the sentence-level representation. Second, inter-sentence and intra-sentence attention are utilized to obtain a representation of the set of sentences containing the target entity.

Research background and motivation

A major bottleneck in the large-scale application of relation extraction tasks is the acquisition of corpus. The distant supervision model automatically constructs large-scale training data by applying the knowledge base to unstructured text alignment, thereby reducing the dependence on artificially constructed data and enhancing the cross-domain adaptability of the model. However, in the process of constructing a corpus using remote supervision, only entity names are used for alignment, and different entities should have richer and more diverse semantic representations under different relationships, resulting in problems such as mislabeling. Therefore, a richer set of entity representations is particularly important.

On the other hand, methods based on grammatical information usually act on the relationship between two entities, and grammatical information can enrich the representation of entities. Therefore, this paper enriches the semantics of entities in different relational schemas based on entity representations in syntactic contexts, and combines neural network models to handle relational extraction tasks.

Related job introduction

We roughly divide related work into two categories: early methods based on distant supervision and recent years based on neural network models.

In order to solve the problem that the relation extraction task relies heavily on the annotated corpus, Mintz et al. (2009) took the lead in proposing a method based on remote supervision to construct annotated corpus. However, the automatically annotated corpus constructed in this way contains a lot of noise. In order to alleviate the impact of noise in the corpus, Riedel et al. (2010) regarded relation extraction as a multi-instance single-category problem. Further, Hoffmannet al. (2011) and Surdeanu et al. (2012) adopted a multi-instance multi-category strategy. At the same time, the shortest dependency path is adopted as a grammatical feature of the relation. The typical drawback of the above methods is that the performance of the model depends on the design of the feature template.

In recent years, neural networks have been widely used in natural language processing tasks. In the field of relation extraction, Socher et al. (2012) adopted recurrent neural network to deal with relation extraction. Zeng et al. (2014) built an end-to-end convolutional neural network. Further, Zeng et al. (2015) assumed that at least one instance in multiple instances correctly represented the corresponding relationship. Compared to assuming that there is an instance representing the relationship between a pair of entities, Line et al. (2016) uses the information contained in the annotation corpus more fully by selecting positive instances through the attention mechanism.

Most of these neural network-based methods use word-level representations to generate sentence vector representations. On the other hand, the representation based on grammatical information has also been favored by many researchers, the most important of which is the shortest dependency path (Miwa and Bansal (2016) and Cai et al. (2016)).

main method

First, based on the dependency syntax tree, a tree-based recurrent neural network (Tree-GRU) model is used to generate a sentence-level representation of entities. As shown in the figure above, instead of just using the entity itself, we can better express long-distance information. The specific entity semantic representation is shown in the figure below. We use Tree-GRU to obtain the semantic representation of entities.

Second, a child-node-based attention mechanism (ATTCE, top figure) and a sentence-level entity representation attention mechanism (ATTEE, bottom figure) are utilized to mitigate the negative effects of syntactic errors and mislabeling.

Experimental results

This paper conducts experiments on the NYT corpus. The end result is shown in the image above. Among them, SEE-CAT and SEE-TRAINS are two strategies used in this paper to combine three kinds of vector representations (vector representation of sentence, vector representation of two entities) respectively. As can be seen from the figure, the proposed model achieves better performance than existing distant supervised relation extraction models on the same dataset.

Summarize

The experimental results in this paper show that a richer semantic representation of named entities can effectively help the final relation extraction task.