SOTAVerified

Crowdsourced Text Aggregation

One of the most important parts of processing responses from crowd workers is aggregation: given several conflicting opinions, a method should extract the truth. This problem is also known as truth-inference in crowdsourcing. Text aggregation problem is dedicated to extracting the correct information from crowd workers' responses for a crowdsourcing task where the output is a text: audio transcription, translation, character recognition, etc.

Papers

No papers found.

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1RASAWord Error Rate (WER)8.6Unverified
2HRRASAWord Error Rate (WER)8.59Unverified
3ROVERWord Error Rate (WER)7.29Unverified
#ModelMetricClaimedVerifiedStatus
1RASAWord Error Rate (WER)15.67Unverified
2HRRASAWord Error Rate (WER)15.66Unverified
3ROVERWord Error Rate (WER)13.41Unverified