Would you be willing to pay more for the last-minute execution of plans just because you didn’t chalk out a few details in advance? It hurts to shell out extra for something that could have been easily well planned in advance. Planning is an indispensable part of our daily lives. The same is applicable when it comes to software development and testing. Planning is the first step forward and selecting the right approach makes all the difference.
Testing is an integral part of software development and planning it well is half the battle won. The same is deemed true for test case designing and execution, which are the founding stones of software testing.
A test case is a well-defined specification of inputs, test procedure, test conditions, and expected output.
Every test case maps to a business requirement and every requirement can have multiple test cases corresponding to it. The more complex the requirement, the higher is the number of test cases corresponding to it. But, is it mandatory to execute every single test case designed to test a functionality? Not really!! The key is to find the right ones, especially when the time is of the essence and resources are scarce or limited
Test case prioritization
The pace at which development is done in the agile methodology, testers are usually left with a limited amount of time in the sprint to go through an entire round of testing. It calls for choosing and executing the right test cases to conduct and effective testing. To that effect, the testing team needs to employ a proper test execution strategy based on two key parameters – severity and priority.
Priority is essentially the order in which defect should be fixed and deals with scheduling.
Severity is the degree of impact on the functionality of the end product.
The following diagram represents defect severity and priority as the two key parameters. Based on the weightage of priority and severity, the right test cases are identified and marked for execution.
| Test case |
Explanation |
Example |
| Critical |
The test cases that need to be executed first are the ones attached to high priority and high severity. |
User’s inability to log in to a system.
Everything stands still if a user cannot log in thus making it critical to fix it first. |
| High |
These are test cases with low priority but with high severity. |
“Change Password” link in the application not working properly.
This is a rarely used feature so the low frequency of accessing this link makes a low priority.But a functionality not working makes it a high severity. |
| Medium |
Medium level test cases determine that the crucial information available on the product is correctly conveyed. |
Incorrect logo/spellings of an organization on every page of a website.
This is a high priority because it is a showcase of the website. However, it is not impacting any functionality thus ranking lower on the severity meter. |
| Low |
These test cases will not impact the functionality or the quality of the product to be released and need not necessarily be executed in case of a time crunch. |
Minor UI mistakes like color and font. |
Identifying the right test cases and test approach
Multiple test cases can correspond to a single requirement, resulting in a big test case repository. Picking which ones to execute, at what stage, is a herculean task.
Test scripts contain multiple steps sometimes running into many lines of code. A prudent and wise approach is to adopt modularity from the beginning itself. It helps in executing specific sections/ test modules, instead of a large script.
Also, it is important to check whether all the identified test cases should be executed or just a subset needs to be considered. It largely depends on the stage at which the testing is being conducted.
The following figure shows the number of test cases that should be ideally executed depending on the stage of development.
Assuming, Timeline for production = 2 weeks and Number of test cases = 1000
Note: These are indicative numbers and can vary based on the application under test.
- At the development stage, execute at least 100 automated test cases for the affected code, several times a day.
- At the integration stage, which may happen maybe twice or thrice a week, approximately 300 test cases may be executed. It is a good idea to consider cases impacting the security posture and overall performance. At this stage, using a healthy mix of manual testing and automated test cases is a preferred approach.
- At deployment, a complete regression testing cycle should be conducted with all the testcases, 1000 test cases in this example.
Needless to say, when the timeline is tight, then pick up the critical and high-impact test cases. For more information: Manual Testing