Cross-Language Assessment of Mathematical Capability of ChatGPT
Gargi Sathe, Aneesh Shamraj, Aditya Surve, Nahush Patil, Kumkum Saxena
Unverified — Be the first to reproduce this paper.
ReproduceAbstract
This paper presents an evaluation of the mathematical capability of ChatGPT across diverse languages like Hindi, Gujarati, and Marathi. ChatGPT, based on GPT-3.5 by OpenAI, has garnered significant attention for its natural language understanding and generation abilities. However, its performance in solving mathematical problems across multiple natural languages remains a comparatively unexplored area, especially in regional Indian languages. In this paper, we explore those capabilities as well as using chain-of-thought prompting to figure out if it increases the accuracy of responses as much as it does in the English language and provide insights into the current limitations.