SOTAVerified

Lexical Normalization

Lexical normalization is the task of translating/transforming a non standard text to a standard register.

Example:

new pix comming tomoroe
new pictures coming tomorrow

Datasets usually consists of tweets, since these naturally contain a fair amount of these phenomena.

For lexical normalization, only replacements on the word-level are annotated. Some corpora include annotation for 1-N and N-1 replacements. However, word insertion/deletion and reordering is not part of the task.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 26–47 of 47 papers

TitleStatusHype
The Denoised Web Treebank: Evaluating Dependency Parsing under Noisy Input Conditions—0
Towards Shared Datasets for Normalization Research—0
To What Extent Does Lexical Normalization Help English-as-a-Second Language Learners to Read Noisy English Texts?—0
Tweet Normalization with Syllables—0
Accurate Word Segmentation and POS Tagging for Japanese Microblogs: Corpus Annotation and Joint Modeling with Lexical Normalization—0
USZEGED: Correction Type-sensitive Normalization of English Tweets Using Efficiently Indexed n-gram Statistics—0
TweetNorm\_es: an annotated corpus for Spanish microtext normalization—0
A Character-level Ngram-based MT Approach for Lexical Normalization in Social Media—0
A Large Corpus of Product Reviews in Portuguese: Tackling Out-Of-Vocabulary Words—0
A Log-Linear Model for Unsupervised Text Normalization—0
An In-depth Analysis of the Effect of Lexical Normalization on the Dependency Parsing of Social Media—0
A Taxonomy for In-depth Evaluation of Normalization for User Generated Content—0
A Text Editing Approach to Joint Japanese Word Segmentation, POS Tagging, and Lexical Normalization—0
A Weakly Supervised Data Labeling Framework for Machine Lexical Normalization in Vietnamese Social Media—0
CL-MoNoise: Cross-lingual Lexical Normalization—0
Contrastive String Representation Learning using Synthetic Data—0
Enhancing BERT for Lexical Normalization—0
Handling Normalization Issues for Part-of-Speech Tagging of Online Conversational Text—0
IHS\_RD: Lexical Normalization for English Tweets—0
Lexical Normalization for Code-switched Data and its Effect on POS-tagging—0
Lexical Normalization of User-Generated Medical Text—0
Multilingual Sequence Labeling Approach to solve Lexical Normalization—0
Show:102550
← PrevPage 2 of 2Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MoNoiseAccuracy87.63—Unverified
2Syllable basedAccuracy86.08—Unverified
3TextNormAccuracy83.94—Unverified
4unLOLAccuracy82.06—Unverified