How to execute manual tests with Xray (Jira)
Lynqa Testing
Introduction
Manual testing remains a key activity in most software projects, even in highly automated environments. There are always critical moments where manual validation is required: before a release, during a regression phase, when validating a complex business rule, or when exploring new or unstable features.
In these situations, manual tests act as a safety net. They help ensure that what is delivered actually works from a user and business perspective, not only from a technical one. This is especially true just before a production release, where confidence in the product is essential and where test results must be clearly visible and traceable.
Having a dedicated tool to structure, execute, and track manual tests makes a real difference. This is where Xray, integrated directly into Jira, becomes extremely valuable. It provides testers clear execution support, allows teams to share a common testing language, and ensures that test results are properly documented and linked to requirements, defects, and releases.
In this article, I’ll walk through how to execute manual tests with Xray. The focus will be on manual execution, and some practical tips based on real-life usage.
Xray Test Types Overview
In Xray, tests can be created with different natures depending on how they are executed:
- Manual Tests
These tests are executed manually by a tester through the Xray interface. They are composed of steps (actions and expected results), and can include parameters and datasets. - Cucumber Tests
These tests are written using Gherkin syntax (Given / When / Then) and are usually executed automatically via CI/CD pipelines. Xray stores and reports the execution results. In some organizations, manual tests are written in Xray as cucumber tests for a possible future automation. - Generic Tests
Generic tests are used when the test logic is implemented outside Xray (e.g. performance tests, security tests, or custom frameworks). Xray only tracks the execution status and evidence.
We will mainly focus on manual test execution, assuming test cases have already been conceived.
Step-by-Step: Execute manual tests with Xray
Follow these steps to execute a manual test with Xray:
1. Create a Test
Navigate to Jira and create a new issue of type Test, or open an existing one. You can access your tests directly from a board or use the Xray test repository. Set test type to “Manual”.
You can now use Add Step to choose between adding a new step, importing an existing one or call an other test as a step for the current one.
2. Create or open a Test Execution
Navigate to Jira and create a new issue of type Test Execution, or open an existing one.
3. Add tests to the Test Execution
Use the Add Tests option to include one or more Test issues.
Tests can be added manually here, or directly from the test repository. A list of the Tests added to this Test Execution appears.
4. Start the execution
Click on Execute to open the manual execution for a dedicated test.
You should now be able to browse the test and proceed to the execution
5. Execute test steps
For each step:
- Perform the described action.
- Compare the actual result with the expected result.
- Set the step status (Pass / Fail).
6. Add comments or evidence
Attach screenshots, logs, or notes to provide execution evidence.
7. Set the overall test status
Once all steps are executed, Xray automatically computes the test status, or you can adjust it if needed.
8. Save the execution
Save the execution to persist the results in Jira, the you can navigate to other tests to execute those.
Using Datasets and Parameters
Xray allows you to reuse the same test logic with different data by using parameters and datasets.
- Parameters
Parameters are variables defined in test steps using a placeholder syntax (e.g. <username>, <password>). - Datasets
Datasets provide concrete values for parameters. Each dataset represents a different data combination.
During execution:
- Select the dataset to use before starting the test.
- Xray will automatically replace parameters with dataset values in the test steps.
- Each dataset execution is tracked separately, improving coverage without duplicating tests.
This makes your test suite smaller, cleaner, and much easier to maintain, especially when validating business rules with multiple data combinations.
Common Mistakes and Best Practices
Common Mistakes
- Executing tests directly without using a Test Execution issue.
- Duplicating tests instead of using parameters and datasets.
- Forgetting to attach evidence for failed steps.
- Mixing manual and automated test logic in the same test type.
Best Practices
- Always execute tests through a dedicated Test Execution issue.
- Use parameters and datasets to avoid test duplication.
- Keep manual test steps clear, atomic, and easy to follow.
- Attach evidence especially for failed or blocked tests.
- Regularly review and clean up obsolete tests.
How to Speed Up Test Execution in Xray
Manual test execution can become time-consuming as test suites grow. Several approaches can help significantly reduce execution time:
Optimize test design
- Use parameters and datasets instead of duplicating similar tests.
- Focus on business-relevant scenarios and avoid overly detailed steps.
Reuse test assets
- Share common test steps and data across multiple tests.
- Group tests logically using Test Plans and Test Sets.
Leverage automation when possible
- Convert stable and repetitive manual tests into automated Cucumber or Generic tests.
- Integrate automated executions into CI/CD pipelines.
Use smart test execution tools
- Tools like Lynqa by Smartesting can automatically execute manual tests from Xray test execution cards, and provide step and test statuses, with clear evidences.
Conclusion
Manual testing is still an essential part of quality assurance, especially at critical stages such as pre-release validation or business acceptance testing. Xray provides a solid framework to organize, execute, and track these tests directly within Jira, making test execution both visible and reliable.
By structuring manual tests properly, using parameters and datasets wisely, and relying on Test Executions as a central execution point, teams can improve both efficiency and confidence in their releases. Combined with smart execution tools like Lynqa and automation where appropriate, manual testing becomes not a bottleneck, but a valuable asset in delivering high-quality software.
FAQ
How do you execute manual tests in Xray for Jira the recommended way?
The recommended method to execute manual tests in Xray is to run them through a Test Execution issue, where you add one or more Test issues and track all runs, statuses, and evidence centrally.
What are the essential steps to execute manual tests in Xray?
Create/open a Manual Test, create/open a Test Execution, add the Tests, execute the run, set step status (Pass/Fail), add comments/evidence if needed, confirm the overall status, and save the execution.
Can you execute multiple manual tests in one Xray execution?
Yes. A single Test Execution can include one or many Test issues, allowing you to execute a full set of manual tests in a single execution container.
How does Xray handle step status vs overall test status during manual execution?
In manual execution, you set each step status (Pass/Fail), and Xray then computes the overall test status automatically.
How do you add evidence when you execute manual tests in Xray?
During the run, you can attach screenshots, logs, or notes as execution evidence; this is especially useful when a step fails.
What are Xray parameters and datasets for manual tests?
Parameters are placeholders in test steps (e.g., <username>, <password>), and datasets provide the real values for those parameters so you can reuse the same manual test logic with different data.
How do datasets work when executing manual tests in Xray?
Before starting the run, select the dataset; Xray will replace parameters with dataset values inside the test steps automatically.
Does each dataset create a separate execution result in Xray?
Yes. Each dataset execution is tracked separately, which increases coverage without duplicating manual tests.
What are the most common mistakes when executing manual tests in Xray?
Common mistakes include executing without a Test Execution issue, duplicating tests instead of using datasets/parameters, forgetting evidence for failed steps, and mixing manual and automated logic in the same test type.
How can you speed up manual test execution in Xray as suites grow?
Use datasets/parameters to avoid duplication, keep steps clear and atomic, reuse assets, group tests using Test Plans and Test Sets, and automate stable tests (Cucumber/Generic) where possible; tools like Lynqa can also help execute and capture statuses/evidence.
You could find this articles interesting:
Stay tuned!
The Future of Test Management
AI Lynqa Testing
Testing in the V-Model: only for mountaineers I like the V-model. Not necessarily the procedure, but the…
Read the article
Discover Lynqa, the AI agent that executes your manual tests automatically
AI Lynqa
A new approach to manual testing At Smartesting, we strongly believe that QA teams deserve more than…
Read the article
Collaborating with a Generative AI: Dream or Reality?
AI Testing
Since the rise of generative AI in our daily lives, software companies have been relentlessly exploring concrete…
Read the article