Language Identification

Language identification is the task of determining the language of a text.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 101–125 of 794 papers

Title	Date	Tasks	Status	Score
AfriHuBERT: A self-supervised speech representation model for African languages	Sep 30, 2024	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	5
Automatic Language Identification in Texts: A Survey	Apr 22, 2018	Language IdentificationSurvey	CodeCode Available	5
Enhance Language Identification using Dual-mode Model with Knowledge Distillation	Mar 7, 2022	Knowledge DistillationLanguage Identification	CodeCode Available	5
MERLIon CCS Challenge: A English-Mandarin code-switching child-directed speech corpus for language identification and diarization	May 30, 2023	Language Identification	CodeCode Available	5
From N-grams to Pre-trained Multilingual Models For Language Identification	Oct 11, 2024	Language IdentificationXLM-R	CodeCode Available	5
Multi-label Scandinavian Language Identification (SLIDE)	Feb 10, 2025	Language IdentificationSentence	CodeCode Available	5
Aggressive Language Identification Using Word Embeddings and Sentiment Features	Aug 1, 2018	Aggression IdentificationBIG-bench Machine Learning	CodeCode Available	5
My Boli: Code-mixed Marathi-English Corpora, Pretrained Language Models and Evaluation Benchmarks	Jun 24, 2023	BenchmarkingHate Speech Detection	CodeCode Available	5
Discriminating Between Similar Nordic Languages	Dec 11, 2020	BIG-bench Machine LearningLanguage Identification	CodeCode Available	5
Offensive Language Identification in Greek	Mar 16, 2020	Language Identification	CodeCode Available	5
On the Language Neutrality of Pre-trained Multilingual Representations	Apr 9, 2020	Language IdentificationTransfer Learning	CodeCode Available	5
On the Language-specificity of Multilingual BERT and the Impact of Fine-tuning	Sep 14, 2021	Language IdentificationNatural Language Inference	CodeCode Available	5
Discriminating between Similar Languages using Weighted Subword Features	Apr 1, 2017	Language IdentificationText Categorization	CodeCode Available	5
Distilled Non-Semantic Speech Embeddings with Binary Neural Networks for Low-Resource Devices	Jul 12, 2022	Emotion RecognitionKeyword Spotting	CodeCode Available	5
Predicting the Type and Target of Offensive Social Media Posts in Marathi	Nov 22, 2022	Language Identification	CodeCode Available	5
problemConquero at SemEval-2020 Task 12: Transformer and Soft label-based approaches	Jul 21, 2020	Language Identification	CodeCode Available	5
Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathi	Sep 8, 2021	Language IdentificationTransfer Learning	CodeCode Available	5
Cross-Domain Adaptation of Spoken Language Identification for Related Languages: The Curious Case of Slavic Languages	Aug 2, 2020	DiversityDomain Adaptation	CodeCode Available	5
CyberTronics at SemEval-2020 Task 12: Multilingual Offensive Language Identification over Social Media	Dec 1, 2020	Feature EngineeringLanguage Identification	CodeCode Available	5
Short Text Language Identification for Under Resourced Languages	Nov 18, 2019	Language Identification	CodeCode Available	5
DocLangID: Improving Few-Shot Training to Identify the Language of Historical Documents	May 3, 2023	Few-Shot LearningLanguage Identification	CodeCode Available	5
Towards Offensive Language Identification for Tamil Code-Mixed YouTube Comments and Posts	Aug 24, 2021	Language IdentificationTransfer Learning	CodeCode Available	5
Comparing the Performance of CNNs and Shallow Models for Language Identification	Apr 1, 2021	Dialect IdentificationLanguage Identification	CodeCode Available	5
Word-level Embeddings for Cross-Task Transfer Learning in Speech Processing	Oct 22, 2019	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	5
Code-Switched Language Identification is Harder Than You Think	Feb 2, 2024	Language IdentificationSentence	CodeCode Available	5

Show:10 25 50

← PrevPage 5 of 32Next →

All datasets VOXLINGUA107 GlotLID-C Nordic Language Identification OpenSubtitles Universal Dependencies VoxForge

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	wav2vec 2.0 LV-60K	Error rate	7.2	—	Unverified
2	XLS-R	Error rate	5.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GlotLID	Macro F1	0.98	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	FastText	Accuracy	0.97	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Apple bi-LSTM	Accuracy	91.37	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Apple bi-LSTM	Accuracy	86.93	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ConformerG-P	Accuracy	99.8	—	Unverified