Alignment based Sequence Ensemble with Multiple Results from a Single Neural Model Architecture

Abstract

Sequence labeling is a fundamental framework that provides the elemental structure and content information for additional natural language processing. However, existing proposed ensemble approaches do not focus on sequence alignment during post-processing. Here, we present a weighted ensemble technique using a sequence alignment approach for a sequence labeling task. Our proposed technique addresses two problems of ensemble technique. First, an ensemble technique requires a multiple model system, and we used a model with multiple random seeds and an additional dropout layer to build a simple ensemble system. Second, we focused on a sequence label list that increased accuracy using alignment for the ensemble technique during post-processing. We evaluated our approach on a representative sequence labeling tasks, part-of-speech and dependency parser. Most results obtained by ensemble sequence alignment approach with various sub-sequence units showed an increase in the F1-score over a single neural network result in the sequence labeling task. Comparing with the hard voting result on the Penn-treebankmarcus1993PTB, the F1-scores increased up to 0.45 at POS-tagged dataset, and up to 0.12 at DP-tagged dataset.

Tasks

Reproductions