Mistakes to Avoid When Verifying Bug Fixes
One of the common activities carried out during software development/maintenance is raising bug tickets for developers to fix whenever there is a deviation from the expected output or behaviour of a functionality. A root cause analysis is conducted to identify the reason for the anomaly and proffer the most optimal and effective solution to fix the bug.
Easy peasy, right? Unfortunately, verifying fixes is much more than that and sometimes we make the assumption of thinking it’s not. Here are five mistakes that should be avoided when verifying a bug fix.
Diving straight into verification without doing some ‘homework’.
There are occasions where bugs are raised by another member of the team. In situations like these, it is very important to know the exact behaviour of the broken build/version, the expected behaviour and conditions that caused the bug to happen. Some deliberate actions that shouldn’t be omitted are:
- Making sure to reproduce the bug on an old build/release. This involves following the steps in the bug report and attempting to reproduce accordingly. It is important to make your personal observations, see the severity of the bug and have at least a rough idea of the cause.
- Paying attention to preconditions: we cannot ascertain that a bug will no longer occur if we don’t reproduce the steps in the exact conditions and context that gave rise to the bug in the first place. Hence, the need to pay attention to such details.
- Deciding before testing, the most effective test strategy. Before attempting to verify the bug fix, it’s good to take a step back to analyse the bug report and ask some questions. What exact test effort is required? Do I need to run tests on only the UI or both UI and APIs? etc. It is very easy to wrongly assume the complexity of a bug and then employ the wrong test efforts.
Forgetting to validate that the bug doesn’t occur in other similar features.
It is okay to just go ahead to reproduce the steps to verify the fix and move on to other tasks but it is wise and a proactive approach to play around similar features or scenarios. Take for example the bug report below:
The tester can test all other login methods and account creation features in offline mode in addition to validating the bug fix.
Failing to check if the fix has resulted in new bugs
A bug fix can result in another bug for many reasons. It could be that the developer who worked on the fix had insufficient knowledge about the feature which was implemented by another developer a while ago and so he was unaware of the intricacies of the design or implementation of the system. A developer may even be required to revisit an old source code he worked on in times past while attempting to fix a bug and sadly, can barely remember the details and reasons for certain implementation. We could go and on about the many reasons why a fix could result in another bug, thus the need to confirm that the fix didn’t result in new bugs.
In this instance, regression tests could be very helpful to confirm that the recent updates to the code base have not adversely affected existing features. It involves re-executing previously prepared test cases to ensure existing functionalities still work. Complete or partial regression tests can be done depending on the scope of the recently updated features. If the team is uncertain of the impact of the bug fix on other modules, then complete regression test is advised.
Having a narrowly focused mindset
Having a narrowly focused mindset can cause us to give a wrong time and effort estimate for development and testing. Bug fixes which require refactoring, for example, seem to be quite easy to fix or test but may not necessarily be, especially when analysing the impact on other modules or the application/system as a whole.
Test effort estimation can be more effectively done when the tester gets adequate information about the scope, nature and the amount of change from the software developer or Architect. It is important not to work in isolation when testing features generally not just bug fixes. For time estimate, it is widely recommended to plan for testing to take much longer than the minimum time you envisaged. This allows for time to cater for unseen and unexpected delays.
Forgetting to keep records
Personally, seeing that I work with a distributed team, I am a huge fan of providing and storing evidence of testing especially when providing feedback to another member of the team. For bug fixes, it is always advisable to document any notable deviation in the functionality of the feature after the fix, how it was fixed (if possible), what strategy was used in verifying, logs and other relevant details such as build/release number. Such records are very valuable in the future when any reference is made to the bug ticket. These records can be saved as a comment on the ticket or feature/bug release notes depending on the structure available in your team.
In conclusion, proper documentation of bug reports is a great way to facilitate correct bug fix implementation and its verification. To this end, great effort should be invested in writing a detailed yet concise bug report that provides as much information as possible that would be helpful to the Software developer and the QA Engineer.
“A problem well put is half solved”― John Dewey.
