Pull Request testing

Shift to the left (real smooth now)

Still zero maintenance

Multiply the hours spent on false-positives and maintaining tests by the number of PRs you ship each day. The cost of all those engineering hours is QA Wolf’s ROI.   

Get your team something nice :)

Parallel testing parallel PRs

Run 100% of your tests in parallel. Then run 100% of your tests in parallel on 100 parallel PRs. Now do it an unlimited number of times each day — without managing test data or causing collisions.

Or we could just do it for you.

Works with your stack

GitHub Actions

GitLab CI

Jenkins

CircleCI

Vercel

Okteto

...and everything else using webhooks

Schedule a demo

The burden of testing multiplied by your velocity

The biggest blocker to continuous delivery is the inability for QA teams to keep pace with the amount of test investigation and maintenance required by product engineering shipping multiple times a day.

For in-house teams, around 15% of end-to-end tests fail each time a suite is run, and investigating those failures can take hours or days. Multiplying that time and cost by the number of pull requests a team merges each day (then adding in the constant data collisions, and the infrastructure to run so many tests each day) and it’s clear why PR testing is only accessible to the biggest and best funded teams.

The value of shifting left (testing as close to the initial development as possible) is obvious: Catching bugs before they’re merged to a shared environment keeps pipelines clear, improves quality, and speeds up delivery.

And QA Wolf makes it possible for any team. If you bring the preview environments to test, we’ll handle the rest.

How it's done

When devs push a PR to preview a webhook calls QA Wolf

These can be static preview environments or ephemeral.

diagram showing how QA Wolf is called via a webhook

diagram showing each test in it's own container managed by it's own k8 cluster

Each test is launched in its own container, and each environment is managed by its own k8s cluster

Isolated tests run faster, flake less, and are easier to review.

PRs are held if tests are failing

While our AI and QA Wolf engineers re-try and update the test scripts.

AI knows when a change was intentional

By accessing the test runner on the preview environment, our AI can decide what to do:

If the change was intentional, the test is updated to match.

If the change was a mistake, a bug report is filed.

FAQs

What is QA Wolf's pull request (PR) testing feature?

Our PR testing feature runs automated end-to-end tests on your pull requests before they are merged. This helps catch issues early, providing immediate feedback to developers and ensuring that only fully tested code makes it into your testing environment and your main codebase.

How does QA Wolf's PR testing work?

When you create a pull request in your SCM, we automatically trigger a deployment of that code to a static or ephemeral preview environment and run tests against it. If any of the tests fail, the PR will be blocked from merging.

Why should I use QA Wolf's PR testing feature?

Because PR testing accelerates your release cycles by catching issues early. Your team saves time and effort that would otherwise be spent troubleshooting the test environment, allowing developers to focus on delivery.

Can QA Wolf handle multiple pull requests at once?

Yes, we can handle multiple pull requests simultaneously. Each pull request triggers its own set of tests in an isolated preview environment, ensuring that every change is thoroughly tested without interference from other ongoing tests. You are given a certain number of concurrent pull requests slots. If your team exceeds the number of concurrent slots, those requests are queued and run once a slot frees up. You can also upgrade to increase your number of concurrent slots.

How do I integrate my repository with QA Wolf?

You can use our native integration to connect GitHub or GitLab repository with a few clicks. Those using other repositories can connect using our API integration. Once connected, we will start monitoring your pull requests and running the necessary tests automatically.

What kind of tests does QA Wolf run on my pull requests?

We run the custom end-to-end tests we build for you. These tests cover critical functionality, edge cases, performance, accessibility, cross-browser compatibility, and anything else you think is necessary to ensure comprehensive test coverage and high-quality code. You can run all of your tests or a customized subset of them.

How do I view the test results?

‍Wherever it’s easiest for you: On the QA Wolf platform, in GitHub, GitLab, or any other SCM you use.

What’s included in QA Wolf’s bug reports?

Repro steps, screenshots, video replays, and browser logs. They’re shared in your team’s messaging app (Slack, Teams, etc.), your issue tracker (Jira, Linear, Asana, etc.) as well as on the PR in your SCM.

What if I need help or support with QA Wolf's PR testing feature?

We offer continuous support and maintenance for your test suite. If you need any updates or modifications, or if you encounter any issues, our team is always ready to assist you.

How does QA Wolf's infrastructure support PR testing?

Our modern infrastructure integrates with popular CI/CD platforms, enabling automated PR testing with immediate feedback. We execute tests fully in parallel to ensure rapid validation of code changes. Additionally, we implement secure, compliant testing practices to protect sensitive data and offer continuous maintenance and support to keep test cases up-to-date and effective. This ensures that every pull request is thoroughly tested, enhancing code quality and accelerating development cycles.

How does QA Wolf's handle flakes during PR testing?

All test suites have up to three chances to pass . Each failing workflow within a suite is retried until it passes or its three chances run out. While the suite is retrying, QA Wolf engineers investigate and attempt to repair it before it runs out of chances, after which it is marked for investigation.