Your automated tests have just finished running. The last deployment only had a few minor changes, so you expect no issues. But alas, three tests failed! You investigate them and discover that when the user clicks the “My Profile” button, nothing happens. Something must have gone wrong with the latest release after all. It’s time to notify the development team ASAP — time to write a bug report!
Situations like the one described above are common for any QA team, whether you’re testing your site manually or with automation. Writing a bug report is more than just a way to document errors, it’s the first — and potentially only — input the developer will receive. It’s also the primary artifact that an engineering team will use to assess severity, risk, and prioritization.
For those reasons, learning to create stellar bug reports is a key skill for any QA team. Good bug reports describe the issue completely but concisely, provide outside context, and connect the issue to any others that were discovered at the same time or in the past. A series of well-written bug reports give the team a big-picture view of the product, and a map showing when and where regressions appear.
Poor bug reporting practices provide none of those benefits. Long, overly-worded reports can hide the root cause of a bug, wasting developer time and energy; while vague and unhelpful reports like “button no longer works” without any context force a developer to do all the investigation themselves. Likewise, when bug reports aren’t connected to each other, or prioritized based on impact or risk of release, engineering teams spin their wheels trying to find the problem or spend time fixing bugs that weren’t going to be released anyway.
In this blog post, we'll delve into the elements of a good bug report that contribute to effective communication and collaboration among development and quality assurance teams.
If nothing else a bug report should have four elements: The expected and actual results of a test case, steps to reproduce the bug, video replays or screencaps of the issue, and browser logs. Together these elements help developers pinpoint an issue, identify the root cause, and fix the problem quickly.
Expected and actual results
This is the essence of a bug report, so clarity is key. Whether you’re doing manual or automated testing, you should have a test matrix or outline that describes any test's expected result. If the actual behavior differs (in other words, if there’s a bug), it’s important to be detailed about what happens and when. It may not always be obvious what caused a bug, so the more detailed the information, the easier for a developer to analyze the root cause.
Repro steps
Some of the sneakier bugs only appear if they’re triggered in a very specific way, that’s why a step-by-step guide outlining the reproduction steps is essential. When a developer doesn’t see the same result that a tester saw, they have to go hunting for additional context — back-and-forth that might take hours — which is time spent blocking a release (best case) or affecting real users (worst case). When writing out repro steps, be descriptive but brief.
Video or screencap
A picture is worth a thousand words, and a video is even better. The additional context helps the developer follow along with the repro steps, and we've found that it's easier to convey tiny details verbally than try to write them out. Since many apps have very similar pages, and different people refer to them in different ways, having a video is like having a GPS instead of a paper map.
Logs
Some bugs can only be found by peeking behind the scenes. The console logs, whether presented within the video or as separate screenshots, provide a technical trail that reveals the errors encountered by the user. These logs serve as valuable clues for diagnosing the root cause of the issue. At QA Wolf we have the HAR files and Playwright traces, which are like DOM snapshots with console logs.
… is understanding the larger context: the affected environment, related test cases, and the severity or business impact of a particular bug. Engineering time is scarce and expensive. These details help PMs, engineering managers, and developers prioritize the time and resources of the team and weigh the cost-benefits of shifting effort from product development to bug squashing.
Severity
Bugs come in different shapes and sizes. Categorizing the severity level — usually low, medium, high, or critical — guides the urgency and priority of bug resolution. Purely cosmetic UI bugs that don't affect the core user experience or impact revenue may not be worth pulling resources from other projects to fix immediately. In contrast, a bug in the authentication flow would prevent anyone from using the product and must be fixed before any other work gets done.
Affected environment
Software behaves differently in various environments. Specifying which environments are impacted by the bug — whether it's all or a specific subset — aids in understanding whether the bug is a widespread issue or confined to specific conditions. This context is pivotal in gauging the severity and scope of the bug. A bug discovered in a local branch shouldn’t necessarily stop the build from graduating to the next environment. But a bug in staging should be blocked from going out to production.
Other tests affected
Frequently a single bug will affect multiple tests in a suite. Identifying them and connecting the dots for the engineering team can help pinpoint the root cause of the issue but it can also change the severity and priority. When a bug blocks a large portion of a test suite from running, the missing test coverage creates opportunities for other bugs to escape. What started as a small P3 problem has the potential to let unrelated P0s escape because the test suite can't check the whole product.
If the name of the game is speed and efficiency, including points of contact and relevant chat threads takes your bug report from an A- to an A+.
Links to communication channels
Bug reporting is all about sharing information. Having a direct line of communication between testers and developers simplifies collaboration
Remedy attempts
These can be included in the video or as a written note on the bug report. They're always helpful to demonstrate the bug in action — and a good double-check for the tester to make sure that the bug is consistently reproducible.
Crafting a comprehensive bug report goes beyond just describing an issue; it's about effective communication and collaboration. By incorporating video demonstrations, clear step-by-step instructions, logs, environmental context, severity assessment, and user interaction details, QA testers can empower developers with the insights to diagnose and resolve bugs swiftly. A well-structured bug report streamlines the debugging process, fosters efficient teamwork, and enhances software quality and user satisfaction.
QA Wolf is a hybrid platform & service that helps software teams ship better software faster by taking QA completely off their plate.
© 2024 QA Wolf, Inc.
