AAA vs GWT: Why QA Wolf prefers narrowly focused test outlines

A quick refresher on AAA and GWT

The Arrange–Act–Assert (AAA) approach is straightforward and to the point.

• Arrange: In this phase, the necessary objects, dependencies, and preconditions for the test scenario are, well, arranged. This might mean generating test data, loading browser extensions, or creating user accounts.
• Act: This is the actual event to test, say uploading a file.
• Assert: Here we verify that the expected outcome did, indeed, occur. If you uploaded a file it should be visible on the file management page.

We go into a lot more detail on AAA here. 

Given–When–Then (GWT) is often called a more “human-readable” approach to test outlining, but as we’ll get to in a moment, the “natural language” approach actually creates blind spots when automating hundreds or thousands of tests. 

• Given: This part describes the initial context, or the state of the system, for a QA engineer to code up. 
• When: The outliner describes an action or event to trigger some behavior to test.
• Then: The expected behavior of the action should be visible or measurable as a result.

The subtle (but important) difference between AAA and GWT

When you look at the two approaches side by side it’s easy to think that AAA and GWT are basically the same. And that’s fair! 

Both frameworks break a test into three parts: A scenario, an event, and a result. There’s a defined input and an expected output. The test passes when the actual result is the same as the expected result. The test fails when it’s not. 

The critical difference between AAA and GWT is their level of detail and, therefore, the scope of what they’re testing. 

AAA encourages high-detail, narrow-scope tests

When you outline a test with the AAA framework, you’ll use extremely rigid definitions at each stage of the test. AAA outlines prescribe exactly how a test should be set up, and exactly what to expect if the test passes. 

For fun and to make this more concrete, let’s outline the purchasing flow on an ecommerce site. 

Arrange:
// in a new context, open the eCommerce site as a customer user
// open a product’s info page

Act:
// select some item quantity
// click ‘Add to Cart’
// nav to the customer’s cart
// click ‘Complete Order’
// fill in shipping information
// click ‘Next’
// fill in payment information
// click ‘Review Order’
// click ‘Submit Order’

Assert:
// customer is redirected to a “Purchase is confirmed” page
// customer receives confirmation email

You’ll notice that this outline specifies just one way that a product can be added to a cart. And because we’ve specified every action that the test will take, if there’s a failure at any step, we’ll be able to isolate it instantly. It takes more time to cover your project with AAA testing, but you’re rewarded with an easier and simpler debugging process down the line.

As an added bonus, when you have to write out the test piece-by-piece, you can see what you’re not testing. We know everything this test covers because we can easily follow the logic. GWT doesn’t provide this luxury.

GWT promotes low-detail, broad-scope tests from the user’s perspective

When you outline with GWT, you’re interested in overall behaviors — not so much the functionality of specific UI elements. 

The GWT test isn’t concerned with how the action is performed, it just checks for an outcome. It could add the item to your cart by clicking a plus sign next to its listing on a larger search page, or it could use the “Add to Cart” button on the item’s unique page. You wouldn’t know which method it used unless you specified, and it’s hard to remember that when you’re constantly testing multiple features.

To show what we mean, look at the same test from above outlined in GWT format.

// Given: That a product was added to the shopping cart, and the user wants to check out.
// When: The user provides payment information and a shipping address, and clicks ‘Complete Order.’
// Then: The product is purchased successfully and the order details are emailed to the user.

AAA tests are interested in discrete pieces of functionality, while GWT tests look at overall system behaviors. A GWT test is satisfied with a positive result no matter how it’s accomplished. An AAA test isn’t satisfied until it works with your precise steps to produce a positive outcome. 

GWT can work well for more generalized use cases. For example, if you’re looking to test if your log-in process works properly every day, then the extra work that AAA requires may not actually help all that much. You just need a yes or no answer, and GWT can provide that.

Why QA Wolf uses the AAA framework

Both AAA and GWT have their place. At QA Wolf, our goal is to provide high levels of test coverage that provide developers with fast, actionable feedback on their code. We’ve found that AAA is a better framework for that goal. 

‍AAA makes it easier to spot gaps in coverage
‍Since the tests are narrowly focused, our QA engineers can systematically check off that every user flow, including the strange edge cases and hard-to-test corner cases are all accounted for. 

If we outlined tests in GWT, we could verify that all the intended user behaviors are tested, but we might miss the functionality of individual components.

‍AAA helps us zero-in on the specific bug
Narrowly written tests that focus on one very specific action let us track down bugs much faster. With more open-ended GWT outlines, a human QA engineer would need to do much more investigation to find the actual bug.

AAA tests are less likely to break and easier to maintain
‍Tests break when developers ship new features and make changes to the UI. The smaller a test’s footprint, the less likely it is to be affected by any single change. This makes them more resilient to frequent updates and simpler to fix when they do break. 

There’s a comprehensive test plan waiting for you!

QA Wolf gets teams to 80%+ end-to-end test coverage in less than 4 months — starting with a comprehensive test plan and detailed outlines for your tests. 

After the 90-day pilot, if you decide QA Wolf isn’t for you, the test plan and test code are yours to keep.