TOOLS COMMONLY USED BY QA TEAMS

QA & Testing activities are carried out according to a process. This process can be more or less mature, more or less formalized, more or less respected, etc. but in any case, there is a process! The process is usually supported by a set of tools.

The purpose of this document is to clarify the role of the different tools commonly used by QA teams in the testing process.

For years, this has been the main and sole professional tool of QA & Testing teams. One product had complete market domination: Mercury Test Director renamed HP Quality Center renamed ALM !  Nowadays, the market offering for Test Management tools is much broader, with tools such as Azure Test Plans by Microsoft, Octane by OpenText, Qtest by Tricentis, Testrail and XRay by Idera, Zephyr by Smartbear and more. Local players such Acqua, Refertest, Squash, Testbench, Xstudio, etc. have also entered the scene. 

A Test Management tool has 2 main functions:

Many Test Management tools also propose 2 additional functions and then are called ALM tools (ex: Helix by Perforce, Polarion, etc.):

Test Management and ALM tools therefore support many tasks of the test process, but not all. In particular, they do not support automatic test execution and obviously, they do not support test design. This is why all our clients use YEST together with a Test Management tool.

Certain heavily regulated industries, such as life-science and medical, aerospace, defense… and other industries that develop highly critical software implement sophisticated requirements engineering strategies that demand more features than what the average Test Management tools can offer. Popular Requirements Engineering tools include Jama Connect, IBM Rational Doors and Rhapsody, etc.

In addition to requirements management, these tools support the other requirements engineering activities such as requirements elicitation, analysis, documentation… These tasks can be carried out using graphical representations and models.

Defining precisely the expected services and performances of the software under development is of great importance for quality assurance. However, writing good requirements is difficult, time consuming and it is considered hardly compatible with agile sprints. In agile contexts, tests can opportunely complement requirements and be an important part of the DoD (Definition of Done)”. This is precisely the purpose of Test Driven Development (TDD) approaches and their variations such as Acceptance Test-Driven Development (ATDD) and Behaviour-Driven Development (BDD).

YEST workflows perfectly fit into these agile approaches, particularly ATDD. YEST workflows indeed represent a first implementation of the requirements, developed collectively by the business representatives (BA and/or PO), the testers and the developers. Agilists refer to this initial test implementation with the expression “specifications by example”. However, this initial test implementation that complements the requirements is also extremely valuable in non-agile contexts.

With agile development practices becoming increasingly popular, the need for development cycles acceleration has led most QA organizations to embrace automatic test execution. Automatic test execution is commonly referred to as “test automation” as if testing was simply about execution and other testing tasks could not be automated as well.

Basically, an automated test execution tool is a robot that commands the execution of scripted sequences on the software application. The offer of such tools on the market is abundant.

Sometimes, test automation engineers are given the manual tests to automate. Therefore, the challenge with automated test execution lies in the coding task and the maintenance of this code. This is where test automation frameworks come into play when they are not part of the tool itself. They propose accelerators and reusable components. Popular frameworks include keyword- and/or data-driven frameworks. 

Recently we have also seen the emergence of automation tools offering a graphics-based framework such as Eggplant or Leapwork. These graphical representations serve a technical purpose, not a functional test design objective. They are low level graphical representations, down to the click sometimes, usually quite far from the business logic. This is why these graphics-based test automation tools cannot, in any way, replace a test design solution like YEST.

Having the required test data right on time is a tough challenge for QA teams. Test Data Management tools such as Delphix, Informatica, Oracle Enterprise Manager… are made to address this challenge. They can usually perform tasks such as crafting data, obfuscating data, creating false data, organizing data…

What they don’t do however, is minimize the number of tests with optimized data combinations. Let’s take the example of a train ticket purchasing platform: your tests will have to be executed with all the possible trip types (one way or return ticket), direct or with connections, all customer categories (infant, young, student, adult, senior), 1st class or 2nd class, membership cards price or list price, etc. Data optimisation techniques exist (pair wise, n wise, etc.) and a couple of tools are available to implement these techniques (Inductive Pairwiser, Hexawise…). YEST offers data combinations optimization service as a native feature among many other game-changing test design services.

Designing test cases that best mitigate risks and really serve the business challenges is a crucial and complex task. Crucial, because it is the source of test quality and justifies or not the subsequent effort in test execution. What is the point indeed to execute and automate test cases if they are not business relevant and smart? Complex, because it requires a good understanding of the specifications and requirements, discipline and organization, collaborative and communication skills, business sense and test technicality capabilities, and creativity. And yet, none of the tools categories presented above truly support test design and help to apply test design techniques.

You will therefore have to consider Model-Based Testing (MBT) tools. MBT is a test design solution that consists in developing a model of the expected behavior of the application under test and letting a tool automatically generate the tests from this model. The model is the sole source of truth and is usually graphical, using notations such as UML or BPMN. MBT has been existing for more than 20 years and several tools have been available ever since: GraphWalker, MaTeLo by AllforTec, MBTsuite by Sepp-Med, Conformiq, among others. However these tools have not had the expected breakthrough for 2 main reasons: 

With YEST, Smartesting has revisited and rejuvenated the MBT approach and addressed the 2 above challenges. Yest graphical representations are easy to develop because they represent test scenarios at a business -level. As such, they are a first step toward the production of executable test cases. YEST also generates tests automatically, but the generated tests can be fine-tuned manually without modifying the graphical representation. As a result, YEST offers the best of both worlds: the support of a rigorous test design approach, the collaborative power of visual representations, and the super-effective production of smart test cases.