Introduction Xray

Issue types used for Testing:

Test

Test should be created within the ticket of a story. It represents one test case, with one or more steps. A story should have at least one Test linked to it, but it could be more depending on the story. These Tests should be added to a Test Plan at the start of a sprint. A Test also has different statusses:

BackLog: added to a story but not yet defined
To Do: the sprint has started, the requirements are clear, so the Test could be picked up to create the test steps, even if development has not yet started
In Progress: the story is being developed or even development is done. The Test should be created and defined. Steps should be described and the expected results should be clear
Done: the Test itself is finished and ready to be executed as soon as the development of the story is done and reviewed and the ticket is set on 'Ready for test"

The Test itself should not be linked to a sprint, but the Test Plan of which this test is a part of should be.

The naming of the Test should reflect the naming of the corresponding story, but edited with the word "Test" as to make clear this is specially for testing. For example, if the title of the story is "Improve the interface" the Test should be named "Test Improve the interface"

Pre- Condition

Pre-Conditions are generic steps that are not necessarily part of the actual Test case, but mostly used as some setup conditions before the real Test can start. These steps are reusable and may be linked to multiple steps. These pre condition should never be linked to any sprint.

Test Plan

To be created at the start of each sprint and linked to that sprint. It is just needed to create an empty Test Plan, which can be filled with Tests created for each story in the sprint. The name should reflect the naming of the sprint, for example "DMS Test Plan sprint 1", "DMS Test Plan sprint 2" etc.

Test Set

A Test Set is a collection of Tests that can be linked to each other for organizing purposes. These can be matched by subject or a specific part of the application. Using Test Sets can make looking up and re-using existing tests more easy and manageable. A Test can be linked to multiple Test Sets. It should be clear by naming the Test Set what the common factor of these Tests are, for examle "Test Set - parts master interface (DLITMPRT)" for all Tests regarding the parts master interface, regardless which story. A Test Set should not be linked to any sprint.

Test Execution

Just like a Test Plan should a Test Execution always be linked to a sprint. Ideally the Test Execution should contain all Tests for every story in the current sprint. At this point just one Test Execution ticket is created for executing all Tests. It is possible to create that Test Execution ticket from the Test Plan ticket:

Atlassian > XRAY (Test Management tool) > image2022-5-11_15-8-9-.png

From the Test Execution it is clear which Tests has been executed, either successfully or not, and which Tests are still to be executed. Where the Test Plan is within the initial phase of the sprint, the Test Execution should be at the the end of the sprint after the story has been developed and the according Tests have been created.

The naming of this Test Execution ticket should have a link to the sprint just like the Test Plan. Example: for Test Plan of sprint 1 it should be something like "Test Execution for Test Plan Sprint 1". At a later stage it could be more preferable to have more Test Execution tickets.

Sub Test Execution

Repositories (PreRequisites)

Do we need / want repositories. If yes, what status, what naming convention etc.

Xray: Testing Process workflow within the Sprint YIS Project

Step by step guide

1. Create a new empty Test Plan

Atlassian > XRAY (Test Management tool) > image2022-5-11_15-34-48-.png

At this moment there will be no Tests yet in this Test Plan. When scrolling down this Edit screen, it is possible to add this Test Plan to the current screen:

Atlassian > XRAY (Test Management tool) > image2022-5-11_15-36-37-.png

2. Create one or more Tests for each story on the board

For each story in this sprint it is possible to create one or more Tests.

Atlassian > XRAY (Test Management tool) > image2022-5-11_15-41-12-.png

This will open up a pop up screen where various setups can be created. At this point the naming is important (a reference to the actual story), checking that the the linked story is correct (under Link Issues) and the the Test Plan is added (under Test Plans):

Atlassian > XRAY (Test Management tool) > image2022-5-11_15-44-3-.png

Optionally a Test Set can be linked here, for example to link this Test to other tests for the same interface. Another option is to add Pre Conditions, that should already have been pre defined.

This test should not be linked to the sprint, so make sure the correct Test Plan is added in order for this Test to be executed once the development of the story is done.

Xray: Testing Process workflow within the Sprint

Step by step guide

When the new Sprint is started TM creates Test Plan for this sprint in Jira

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 16.47.04.png

TM creates Tests (issue type) for all stories included into Sprint and associates them with Test Plan.

Tests should be created from the story:

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 16.52.11.png

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 16.54.38.png

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 16.54.57.png

2. When Tester starts work with the story, he/she creates Test Design for the story within the associated Test

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 17.02.11.png

Tester add test steps with all necessary data and Expected Results

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 17.04.01.png

3. The development is done and story is deployed for testing, Tester goes to the story and creates Test Execution for the newly created Test

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 17.08.43.png

4. To start test execution click icon "View runs" from the Story:

Atlassian > XRAY (Test Management tool) > test step n1.png

or click on "Execute in" in Test issue:

Atlassian > XRAY (Test Management tool) > image2021-9-29_16-4-55.png

Test run will open in a new tab.

If you don't see the "View runs" icons in the Story, open the Test, scroll down to Executions. There is a Play (Run) button near each execution.

Click it and select Execute:

Atlassian > XRAY (Test Management tool) > executions.png

Please note: to be able to run the execution, it should be assigned to you. Otherwise you will not see the Execute button.

5. Tester executes all steps manually on test env and set the status for each step (Pass/Fail) within created Test Execution

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 17.13.39.png

Bugs should be associated with test step if found

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 17.15.33.png

6. Each Test Execution should be added to Test Plan by Tester

Atlassian > XRAY (Test Management tool) > Screenshot 2021-07-13 at 17.18.00.png

7. If bug was found and test step is failed, new Test Execution should be created after bug fix. Steps 3-6 should be repeated

8. Test is completed when all test steps are green for latest Test Execution

Boards

XRAY Test Plan Board

Planning per sprint
Atlassian > XRAY (Test Management tool) > image2022-5-11_13-54-33-.png

Jira Project Board YIS- All test products

Issue type visible on board: Plan and Plan Execution

Use Switch sprint at the top to filter on a specific sprint.

Atlassian > XRAY (Test Management tool) > image2022-5-11_13-57-36-.png