SOTAVerified

Crowdsourced Text Aggregation

One of the most important parts of processing responses from crowd workers is aggregation: given several conflicting opinions, a method should extract the truth. This problem is also known as truth-inference in crowdsourcing. Text aggregation problem is dedicated to extracting the correct information from crowd workers' responses for a crowdsourcing task where the output is a text: audio transcription, translation, character recognition, etc.

Papers

No papers found.

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1RASAWord Error Rate (WER)8.6—Unverified
2HRRASAWord Error Rate (WER)8.59—Unverified
3ROVERWord Error Rate (WER)7.29—Unverified
#ModelMetricClaimedVerifiedStatus
1RASAWord Error Rate (WER)15.67—Unverified
2HRRASAWord Error Rate (WER)15.66—Unverified
3ROVERWord Error Rate (WER)13.41—Unverified