Skill Check: Some Considerations on the Evaluation of Gamemastering Models for Role-playing Games
2023-09-24Code Available0· sign in to hype
Santiago Góngora, Luis Chiruzzo, Gonzalo Méndez, Pablo Gervás
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/sgongora27/skill-check-gm-testsOfficialIn papernone★ 2
Abstract
In role-playing games a Game Master (GM) is the player in charge of the game, who must design the challenges the players face and narrate the outcomes of their actions. In this work we discuss some challenges to model GMs from an Interactive Storytelling and Natural Language Processing perspective. Following those challenges we propose three test categories to evaluate such dialogue systems, and we use them to test ChatGPT, Bard and OpenAssistant as out-of-the-box GMs.