Arrange, Act, Assert: An introduction to AAA

Whether you're a small start-up, a large enterprise, or you partner with companies of all sizes like we do, establishing test writing standards for the QA team ensures that your entire application is covered, the tests are well-structured, and the test suite is more resilient to code changes. We’ve decided to use the Arrange, Act, Assert (AAA) framework because it's flexible, specific, and efficient.

A walk through AAA outlining

We’ll get into the Arranging, Acting, and Asserting in just a bit, but it’ll be helpful if we start by defining a couple other terms we use at QA Wolf:

A test confirms that one event triggered one result. It’s defined in an outline or matrix, and built in code. In this post we’re going to use an example — creating a new slideshow — in which the user uploads photos to their account, creates a new (empty) slideshow, and adds their photos to it.

A workflow is a series of tests that relate to or build on each other. All the steps needed to create a slideshow — logging into an account, creating a new photo slideshow, giving it a name, and finally adding photos to it — are considered a workflow. 

We’re also going to refer to the “user” in this post, which is the automated test-running robot written in code (Playwright in our case) that simulates a human-user’s actions. 

Okay! That’s out of the way. Let’s talk about the AAA framework and how it helps you build a test matrix, outline individual tests, and maintain tests as they break.

Arrange

Before you can test a workflow or bit of functionality, you have to set up — or Arrange — the application a certain way. For instance, to test whether a user can log out, you have to arrange a scenario where the user is logged in. 

It’s very common for the same user action to be tested multiple times and under different arrangements because bugs may appear in some cases but not in others.

When outlining the Arrange step, we recommend straight-forward and brief instructions that reduce the chance of misunderstanding or errors in the test code.

Let’s look at our “Create New Slideshow” example. We want to arrange the test so that two photos are already uploaded to a folder.

We'd outline that like so:

1 // Arrange:
2 // log in with a valid username and password
3 // open the folder titled "Slideshow Photos"
4 // upload photos until there are at least two photos in the folder, if needed

We start by having the user log in to the site with a valid username and password. Next, the user opens a folder or creates one if necessary. Finally, we add photos to the folder so that there’s something for the “Create New Slideshow” test to pull from. 

And just like that we’ve described how the test scenario should be arranged each time.

For this particular test we’re not bothering to “Assert” that the user logged in successfully or the folder was created because both of those would be covered by other tests — remember, a test confirms that one event has one result. By narrowly defining them this way it’s easier to track down the bug when a test fails. 

Act

The “Act” stage is where we describe the event to be tested (one event, one result). For our example, that means creating a new (blank) slideshow and adding two photos to it. Other tests would be created for separate scenarios like “user clicks the ‘Save’ button without adding any photos.” 

Again, these narrowly-focused tests make it easier to track down bugs when tests fail and make tests easier to maintain.

Take a look at the test outline when we add the Act section…

// Arrange:
// log in with a valid username and password
// open the folder titled "Slideshow Photos"
// add photos until there is at least two photos in the folder, if needed

// Act:
// click '+' icon in the lower-right corner of the page
// from the menu, select "Create Slideshow"
// in the name field, enter name in the format "Slideshow [[ any number ]]"
// click the '+' icon
// select one photo (photo A)
// click the '+' icon again
// select a different photo (photo B)
// click 'Save'

When outlining the Act stage, it’s important to be specific so that the QA engineer who writes and maintains the test code is clear about what’s supposed to happen. A couple things you might have noticed in the outline above:

1. Indicating the placement of the + icon, in case there are multiple on the page.
2. Using unique file names. This is particularly important if you run your tests in parallel like we do at QA Wolf to avoid conflicts between multiple tests.
3. Distinguishing between multiple photos. Again, to make sure that the QA engineer can work quickly and independently.

Clearly documenting your outlines up front will save you a lot of time in the long run.

Assert

The last A in AAA is “Assert” and it’s arguably the simplest because it’s the expected result. In our example, that means that a slideshow was successfully created and that a confirmation message appears. 

We include that in our outline like this:

// Arrange:
// log in with a valid username and password
// open the folder titled "Slideshow Photos"
// add photos until there is at least two photos in the folder, if needed

// Act:
// click '+' icon in the lower-right corner of the page
// from the menu, select "Create Slideshow"
// in the name field, enter name in the format "Slideshow [[ any number ]]"
// click the '+' icon
// select one photo (photo A)
// click the '+' icon again
// select a different photo (photo B)
// click 'Save'

// Assert:
// slideshow has been created with the given name and photos
// a confirmation message appears (and disappears) reading "New slideshow created"

The text in the “Assert” stage describes the intended outcome of the workflow. In this example, that was as simple as checking that the created slideshow was visible and that a confirmation message appeared and disappeared with relevant text.

● ● ●

And there you have it! With the “Create New Slideshow” we’ll always know if the feature is working or if a regression was introduced by a change. When the test fails, it’ll be easy for the QA engineer to track down the cause or repair the test based on the AAA documentation. 

Why AAA?

There are three big reasons why our QA engineering team likes the AAA framework for outlining tests:

1. With standardized outlines, both QA engineers and clients can find gaps in coverage and where additional assertions or tests might be needed.
2. You can quickly isolate bugs and communicate them to the engineering team that's responsible for them.
3. If you've thoughtfully documented naming conventions, it's easy for other QA engineers to pick up and understand the test, allowing them to jump into test creation and maintenance much faster.

Here’s an example of how outlining a test with the AAA framework helped our team quickly triage a test and file a bug report for a client: 

This client had a workflow that would assert a user could create and cancel their own schedules on an event planning web app. We created two workflows: “Create a new schedule” and “Cancel a schedule with the ‘Actions’ button.” Within each workflow, we employed AAA methods to write outlines that were focused solely on the feature named by the workflow. The created workflows performed smoothly and were put into production.

A week later, the “Cancel” workflow failed for unknown reasons. A separate QA engineer was assigned to investigate. Because these workflows were written separately with clearly-defined assertions, the QA engineer was able to determine that when the user clicked the ‘Cancel’ button, nothing happened on the screen and error messages appeared in the browser’s Dev Tools console. They were confident that this bug was not affecting anything else because the “Create” workflow continued to pass on every run.

With a few clicks, my teammate had isolated the bug and sent off the bug report, all without having to ask me for additional context.

AAA-rated best practices

There are a few practices that QA Wolf has found really helpful when outlining with AAA. Maybe they’ll be useful to you too.

One step, one Act/Assert pair

As I’ve mentioned throughout this post, AAA works best when there’s just one assertion for every action. Things get really messy if you try to do more than that. For example, say we also wanted to test that additional slides could be added after our slideshow was saved. 

If the test then fails when adding that second slide, you wouldn’t know that the first half of the test — “Create New Slideshow” — worked fine. 

The best thing to do would be create two tests. The first would be “Create New Slideshow” and would Assert that the slideshow was created with the given name. The second workflow would be titled “Add Slide to Existing Slideshow Presentation” and would assert that the slide was added successfully. 

With this division, if there’s a bug with adding a slide to a presentation, it will be isolated from other workflows and make it easier to de-bug.

Specificity over generalization, in general

Say you have a standard Profile Page and want to test the ability to add or change a Profile Image and change the password. 

We would create specific workflows for each of those tasks, like the example below, because we’ll be better able to isolate bugs and test maintenance will be easier. 

Workflow 1: “User Profile Page - Profile Picture”

• Test 1: “Add profile image to user’s profile”
• Test 2: “Change profile image on user’s profile”

Workflow 2: “User Profile Page - Change Password”

• Test 3: “Change user’s password”

Now we also have a major advantage: QA Wolf's back-end has the ability to run all of our tests in parallel. If you're doing this yourself without the QA Wolf you may find it easier to create one generalized workflow like the one below. Especially if the Profile Page is laid out so that Profile Image and Password are all on the same page. It’ll be harder to debug but may be easier to create and run.

Combined workflow: “User Profile Page - Profile Picture and Change Password”

• Test 1: “Add profile image to user’s profile”
• Test 2: “Change profile image on user’s profile”
• Test 3: “Change user’s password”

AAA is flexible. It can be used to combine or split testing scenarios into any number of workflows. The important thing is to be consistent in your structure and  deliberate with your naming conventions.

Be clear about what you're trying to test

And make sure that you're testing that function only once. This is the best way to eliminate redundancy in your test outlines, which reduces the workload for QA engineers and the chance of errors. Here's an example.

Say you want to test a check-out flow on an ecommerce site. Before you can test the check out, you have to arrange the test by finding the product and adding it to the cart. You think to yourself, well, “those are also functions I want to test — why don’t I combine them all into the same workflow since they’re all related?” Well, you could, but now that you're validating each of those events you'll want three separate tests in your workflow and would end up with something like this...

Test 1: Navigate to product’s listing page

// Arrange:
// log in

// Act:
// in search bar, enter “apple”
// click ‘Search’
// click the first item on the list to pull up its details page

// Assert:
// the product’s full name, price, and description are visible
// the ‘Add To Cart’ button is visible and enabled
// the product’s reviews are visible (if there are none, text appears saying so)

‍Test 2: Add product to cart

// Arrange:
// log in
// in search bar, enter “apple”
// click ‘Search’
// click the first item on the list to pull up its details page

// Act:
// click ‘Add To Cart’
// click on the cart icon in the upper-right corner of the page

// Assert:
// a toast message appears reading “Added to Cart!”
// in the user’s cart, the product and its name, price, and quantity are visible and correct
// the ‘Checkout’ button is visible and enabled

‍Test 3: Checkout with one product in cart

// Arrange:
// log in
// in search bar, enter “apple”
// click ‘Search’
// click the first item on the list to pull up its details page
// click ‘Add To Cart’
// click on the cart icon in the upper-right corner of the page

// Act:
// click ‘Checkout’
// fill in shipping information
// click ‘Next’
// fill in card information
// click ‘Preview Order’
// click ‘Submit Order’

// Assert:
// the user is taken to a page with the text “Your order has been successfully submitted!”
// the order’s product details are visible and accurate
// the user receives a confirmation email with the order number in the subject line

Do you see how much repetition there is? Each new assertion requires you to repeat the Act stage of the previous test and make it part of its Arrange stage. If we’re truly only asserting that a product order can be submitted, then tests 1 and 2 are technically a distraction from this main goal. Tests 1 and 2 should instead be covered by different workflows. If we had stayed with our original goal of simply testing the checkout function in this workflow you could have combined all of that into into something like this: 

Test: Checkout with one product in cart

// Arrange:
// log in
// in search bar, enter “apple”
// click ‘Search’
// click the first item on the list to pull up its details page
// click ‘Add To Cart’
// click on the cart icon in the upper-right corner of the page

// Act:
// click ‘Checkout’
// fill in shipping information
// click ‘Next’
// fill in card information
// click ‘Preview Order’
// click ‘Submit Order’

// Assert:
// the user is taken to a page with the text “Your order has been successfully submitted!”
// the order’s product details are visible and accurate
// the user receives a confirmation email with the order number in the subject line

Here’s a good rule of thumb: If one step depends on the previous step to occur, those steps should be combined. This reduces redundancy in the “arrange” section and makes your test as specific as possible. Again, this should only be done if it’s confirmed that any related flows are also being tested elsewhere.

Standardize your language 

It’s helpful for test outliners and QA engineers to have a shared vocabulary. For instance, at QA Wolf we use distinguish between "Click" for buttons and "Select" for menu items because an interface will often have similar labels on different elements. Having these conventions in place is just another way we reduce the chance of confusion or miscommunication when writing tests.

Include clean-up steps in your outline

If a workflow creates something, it should be deleted at the end wherever possible. One reason is that a previous run that failed part-way through may leave data that interferes with the next run. By decreasing the occurrence of “irrelevant data,” you’ll also reduce load times and improve the health of the testing environment. Adding a “add clean-up step” note to the end of outlines (or other reminder) ensures the developer knows to add one to the beginning and/or end of the workflow.

Final thoughts

Creating tests is a vital step in the QA process to ensure bugs are not shipped out to customers accidentally. With AAA, QA Wolf can outline tests quickly for our clients, speed up the engineering, and simplify long-term maintenance.

‍

‍

‍

‍