Discuss and formulate acceptance test scenarios to reinforce requirements elicitation has become a strong agile practice in the recent years.
Agile testing best practices – Gherkin-based BDD versus Visual ATDD
They are called ATDD – Acceptance Test-Driven Development, BDD – Behavior-Driven Development and even sometimes SBE – Specification By Example.
The test practices behind these acronyms have similar philosophy, but different positioning and implementations. In this blog post, we will compare two approaches that are positioned at different levels in the agile cycles of requirement clarification:
- Visual ATDD that uses graphical representations of test workflows to support the discussion and formulation of test scenarios that are positioned at the level of the Feature or Business Process with the objective of producing tests at the application and inter-application levels.
- BDD Gherkin that uses the Given – When -Then text format to define test cases associated with the acceptance criteria of the refined User Story with the aim of producing more atomic User Story-centric tests.
The objective of this article is to compare visual ATDD and BDD Gherkin and to show the similarities and differences in their nature on a concrete example.
VISUAL ATDD and BDD GHERKIN: WHAT IS SIMILAR
These approaches share the same philosophy: the production of test scenarios aims to reinforce the requirements elicitation and the acceptance criteria through use cases examples, which become acceptance tests.
In the ISTQB advanced level syllabus “Agile Technical Tester”, BDD Gherkin is defined as « … a technique in which developers, testers, and business representatives work together to analyze the requirements of a software system, formulate them using a shared language, and verify them automatically ». This definition also applies to visual ATDD !
This definition clearly articulates the key points:
- BDD and ATDD acceptance test scenarios are used to refine the requirement elicitation in Agile under the form of testable examples.
- These acceptance test scenarios formalize what is expected from the software being developed, and they represent examples of the software usage.
- The test scenario is shared and understandable by all members of the Agile team: business analysts, Product Owner, testers and developers.
Another emerging concept of Agile is common to both Visual ATDD and BDD-Gherkin: acceptance test scenarios are a living documentation of the product.
Epics and User Stories are volatile information: when a User Story is implemented in an iteration and accepted after the test run, this expression of the requirement has an historical value, but this cannot make a requirements repository.
Acceptance test scenarios however are bound to last over time, particularly because they are implemented into automated regression tests and often automatically executed. Therefore, they are always up to date and in-synch with the application. They are easy to read and understand, making them an adequate support of communication for technical and non-technical members of the Agile team.
VISUAL ATDD and BDD GHERKIN: THE DIFFERENCES
The format of the scenarios is of course the first difference:
- In visual ATDD, the application workflows are represented graphically by a diagram, describing a test workflow.
- In BDD-Gherkin, the textual format structured in Given – When – Then, describes an individual scenario in terms of test context (the Given part), test action (the When part) and expected result (the Then part).
The main difference clearly appears in these two definitions: in visual ATDD, the test scenario represented by the graphical workflow presents a global vision of the addressed functionality or business process, whereas in BDD-Gherkin, it is about covering the acceptance criteria of a User Story in an atomic fashion.
ILLUSTRATING VISUAL ATDD WITH AN EXAMPLE
Let’s take the example of a company that wants to develop a business travel management function within its HR software.
This project is sponsored by the HR department, which presented the expected functionality during a requirement clarification workshop as follows:
The system should enable
– an employee to create a travel plan,
– his/her manager to validate the created travel plan
– the HR department to proceed to the reimbursement of the travel expenses
This statement corresponds to a “Minimal Marketable Feature”, defined by the Agile Alliance as a minimal functionality that makes sense for the Business. This minimal dimension allows the team to focus on the first value to be delivered, which can then be complemented.
The visual ATDD test workflow for this functionality is presented in Figure 1. This workflow was discussed and formulated during a “3 Amigos” workshop attended by two representatives of the HR department (source of the business requirement), the HR application Product Owner, a tester and two developers from the team. The development of the workflow aims at aligning the whole team on the different functions to be implemented. Acceptance test scenarios and their corresponding automated scripts will directly derive from these test workflows.
The team uses the Personas method (archetypes of users of the system). Three Personas have been defined: Peter is an employee of the company; Emma is a Manager and Joe is a person from the HR department.
Several decision tables are associated with this workflow to refine business rules. For example, if the Manager rejects the request, he will have to give predefined reasons (Too expensive, Trip too long, Other reason), as shown in the following table associated with the task “Validation of the request” in the above workflow.
Test scenarios in visual ATDD are designed to cover the different use cases from the test workflow. The visual scenario, highlighted in green, is an example of a test obtained from the workflow.
Each test case produced from the workflow is formatted as textual scenario and as an automated test script. The test documentation below corresponds to scenario highlighted in the workflow of figure 2.
THE SAME EXAMPLE DEVELOPED WITH BDD-GHERKIN
The Gherkin language is not well-suited to write examples at the global workflow level of an entire application module.
If one tries to write the previous test case in Given – When – Then format, one reaches the readability limits of a Gherkin scenario that becomes too long and complex (due in particular to the change of role between Peter, Emma and Joe). And remember! Our “travel management” example is simplified: in real projects, test workflows are more complex and longer than the one described in this paper. Imagine a BDD Gherkin scenario with more than 15 or 20 test steps! The readability and maintenance of such BDD Gherkin scenarios would be close too impossible.
BDD Gherkin scenarios are meant to write test for the acceptance criteria of refined User Stories, and not for application or inter-application scenarios.
An example of a good use of the BDD Gherkin is to consider a test case at the level of a User Story acceptance criteria. Let’s consider the User Story “Trip Validation/Refusal” which has been defined by refining the requirement for the “Travel Management” functionality. An acceptance criteria for this User Story is the rejection for the “too expensive” reason, for which BDD Gherkin scenario would look as follows:
User Story – Request validation
I can refuse a travel request from a member of my team
to manage my budget
Scenario: rejection of a travel request for the reason « too expensive »
Given I’m connected as Emma
When Peter issues a travel request form
and Peter is in Emma’s team
and Emma rejects Peter’s travel request for the reason « too expensive »
Then Peter’s travel request is rejected with the reason « too expensive »
The principle of the BDD Gherkin is to develop scenarios focused on an acceptance criteria and to produce the corresponding atomic automated code. This BDD Gherkin scenario could be parameterized by a data table giving the different reasons for rejection. We won’t do this parametrization in this paper for concision reasons.
SUMMARY: RESPECTIVE POSITIONING OF VISUAL ATDD AND BDD-GHERKIN
Visual ATDD and BDD Gherkin are positioned at different levels in the Agile cycle of requirements clarification, as illustrated in the following figure:
- Visual ATDD:
o Test workflows stand at the functionality level – Minimal Viable Features.
o Decision tables describe the User Stories at User Story acceptance criteria level.
- BDD Gherkin:
o the Given – When – Then Scenarios are positioned at the level of the User Stories acceptance criteria.
The following figures show the respective positioning of the Visual ATDD product (Figure 5) and the Gherkin BDD (Figure 6) in relation to the backlog.
AUTOMATION WITH VISUAL ATDD AND WITH BDD GHERKIN
Test automation in Visual ATDD and BDD Gherkin follows a similar approach, based on keywords and, if useful, data tables. Test scripts are automatically generated by the supporting tools and the test developers or automation specialists code and maintain the automation keywords.
The following figure shows a Robot Framework script generated with YEST for the scenario in figure 3 of the Visual ATDD.
The CucumberStudio environment for the Gherkin BDD also allows to link the sentences of the scenario with implementation code in different languages (glues or Fixture).
So, which approach to choose: Visual ATDD or BDD Gherkin?
Well it obviously depends on your context and your objectives in implementing Agile testing practices.
If your context is that of applications in enterprise IT systems, and your objective is to define, implement, and test business workflows (or business processes), then the visual ATDD approach is relevant because it allows you to involve business-oriented roles in valuable discussions on application workflows examples. Graphical workflow formalism is familiar to business analysts and functional analysts, and decision tables will help to discuss and detail the different implementation cases of business rules.
The BDD Gherkin is a good approach to refine in detail, with business stakeholders, the acceptance criteria at the level of each User Story, discuss the examples, formulate the scenario and automate the corresponding script.
Depending on your objectives and application context, you may use Visual ATDD for a global view of the Feature and produce application or cross-application tests, and/or use the BDD Gherkin to script tests at the User Story level and produce short, atomic tests at the Acceptance Criteria level.
Smoke Testing, Sanity Testing, and Regression testing: the Trifecta Understanding the differences between Smoke, Sanity, and Regression Testing is crucial…
Historically, software testing was confined to pre-production environments to validate the application against the written requirements. (also known as requirement-based…
The surge of Large Language Models (LLM) like GPT has undoubtedly revolutionized the way we approach natural language understanding and…