Agentic AI for Behavior-Driven Development Testing Using Large Language Models

Code

• github.com/unibuc-cs/BDDTestingWithLLMs
  none★ 2

Abstract

Behavior-driven development (BDD) testing significantly improves communication and collaboration between developers, testers and business stakeholders, and ensures that software functionality meets business requirements. However, the benefits of BDD are often overshadowed by the complexity of writing test cases, making it difficult for non-technical stakeholders. To address this challenge, we propose BDDTestAIGen, a framework that uses Large Language Models (LLMs), Natural Language Processing (NLP) techniques, human-in-the-loop and Agentic AI methods to automate BDD test creation. This approach aims to reduce manual effort and effectively involve all project stakeholders. By fine-tuning an open-source LLM, we improve domain-specific customization, data privacy and cost efficiency. Our research shows that small models provide a balance between computational efficiency and ease of use. Contributions include the innovative integration of NLP and LLMs into BDD test automation, an adaptable open-source framework, evaluation against industry-relevant scenarios, and a discussion of the limitations, challenges and future directions in this area.

Tasks

Reproductions