SOTAVerified

Lexical Normalization

Lexical normalization is the task of translating/transforming a non standard text to a standard register.

Example:

new pix comming tomoroe
new pictures coming tomorrow

Datasets usually consists of tweets, since these naturally contain a fair amount of these phenomena.

For lexical normalization, only replacements on the word-level are annotated. Some corpora include annotation for 1-N and N-1 replacements. However, word insertion/deletion and reordering is not part of the task.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–25 of 47 papers

TitleStatusHype
ViSoLex: An Open-Source Repository for Vietnamese Social Media Lexical NormalizationCode0
A Weakly Supervised Data Labeling Framework for Machine Lexical Normalization in Vietnamese Social Media—0
ViLexNorm: A Lexical Normalization Corpus for Vietnamese Social Media TextCode1
Automatic Textual Normalization for Hate Speech DetectionCode0
Increasing Robustness for Cross-domain Dialogue Act Classification on Social Media DataCode0
A Character-level Ngram-based MT Approach for Lexical Normalization in Social Media—0
CL-MoNoise: Cross-lingual Lexical Normalization—0
MultiLexNorm: A Shared Task on Multilingual Lexical NormalizationCode0
Sesame Street to Mount Sinai: BERT-constrained character-level Moses models for multilingual lexical normalization—0
Multilingual Sequence Labeling Approach to solve Lexical Normalization—0
A Text Editing Approach to Joint Japanese Word Segmentation, POS Tagging, and Lexical Normalization—0
To What Extent Does Lexical Normalization Help English-as-a-Second Language Learners to Read Noisy English Texts?—0
ÚFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5Code1
Contrastive String Representation Learning using Synthetic Data—0
Sequence-to-Sequence Lexical Normalization with Multilingual Transformers—0
DaN+: Danish Nested Named Entities and Lexical NormalizationCode0
User-Generated Text Corpus for Evaluating Japanese Morphological Analysis and Lexical NormalizationCode0
Lexical Normalization for Code-switched Data and its Effect on POS TaggingCode0
Lexical Normalization for Code-switched Data and its Effect on POS-tagging—0
Synthetic Data for English Lexical Normalization: How Close Can We Get to Manually Annotated Data?—0
Norm It! Lexical Normalization for Italian and Its Downstream Effects for Dependency Parsing—0
A Clustering Framework for Lexical Normalization of Roman UrduCode0
Adapting Deep Learning for Sentiment Classification of Code-Switched Informal Short TextCode0
A Multi-cascaded Deep Model for Bilingual SMS ClassificationCode0
Enhancing BERT for Lexical Normalization—0
Show:102550
← PrevPage 1 of 2Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MoNoiseAccuracy87.63—Unverified
2Syllable basedAccuracy86.08—Unverified
3TextNormAccuracy83.94—Unverified
4unLOLAccuracy82.06—Unverified