Judgment of Thoughts: Courtroom of the Binary Logical Reasoning in Large Language Models

Abstract

This paper proposes a novel prompt engineering technique called Judgment of Thought (JoT) that is specifically tailored for binary logical reasoning tasks. JoT employs three rolesx2014lawyer, prosecutor, and judgex2014to facilitate more reliable and accurate reasoning by the model. In this framework, the judge utilizes a highx2010level model, while the lawyer and prosecutor utilize lowx2010level models. This structure helps the judge better understand the responses from both the lawyer and prosecutor, enabling a more accurate judgment. Experimental results on large language model (LLM) benchmark datasets, such as BigBenchHard and Winogrande, demonstrate that JoT outperforms existing methods, including Chain of Thought (CoT) and Selfx2010Consistency (SC), in binary logical reasoning tasks. Additionally, in realx2010world tasks, such as Fake News Detection and SMS Spam Detection, JoT shows comparable or improved performance compared to existing techniques. JoT significantly enhances the accuracy and reliability of models in binary reasoning tasks and show potential for practical applicability across various domains. Future research should aim to further broaden the applicability of JoT and optimize its implementation for realx2010world problemx2010solving.

Tasks

Reproductions