Semi-supervised Multitask Learning for Sequence Labeling

Code

• github.com/marekrei/sequence-labeler
  OfficialIn papertf★ 0
• github.com/MirunaPislar/multi-head-attention-labeller
  tf★ 16
• github.com/samueljamesbell/sequence-labeler
  tf★ 11

Abstract

We propose a sequence labeling framework with a secondary training objective, learning to predict surrounding words for every word in the dataset. This language modeling objective incentivises the system to learn general-purpose patterns of semantic and syntactic composition, which are also useful for improving accuracy on different sequence labeling tasks. The architecture was evaluated on a range of datasets, covering the tasks of error detection in learner texts, named entity recognition, chunking and POS-tagging. The novel language modeling objective provided consistent performance improvements on every benchmark, without requiring any additional annotated or unannotated data.

Tasks

Benchmark Results

Reproductions