THE TRUE COST OF END-TO-END TESTING IN-HOUSE

Step 1
Determine how many tests you need
Developers on your team
-
+
Team size is a good indicator of your app’s testing footprint.
Coverage
High
80%
Aiming for 80% E2E test coverage ensures critical paths are well-tested, significantly reducing the risk of undetected bugs.
Learn more
Step 2
Test creation & management
Time to create each test
2hrs
2 hours per test case on average. This includes outlining, coding, creating test data, and verifying it works.
Time to investigate and fix a failed test
1.5 hrs / test
~1.5 hours on average. This includes re-running the test, reproducing the failure, debugging, PR reviews, and re-deploying the test code.
edit fail rate
Fail rate
7.5%
High velocity teams with daily releases see about 7.5% of tests fail.
QA engineer salary
$90,000 / year
As of January 2022, the base salary for titles like Senior QA Engineer or Senior Test Automation Engineer in the US was $90,000 before benefits and other comp.
Number of SDETs
By default, we recommend the number of Software Development Engineers in Test (SDETs) based on industry standards.

An SDET enhances your QAE team by developing automated testing tools and frameworks, increasing testing efficiency, coverage, and reliability.
SDET Salary
$120,000 / year
As of January 2022, the base salary for titles like Senior Software Engineer in Test and SDET in the US was $120,000 before benefits and other comp.
Step 3
Runs setup
Run Cadence
-
+
You should run the suite at least daily, or more if you deploy continuously.
Run type
Tests run simultaneously across multiple resources. This method can reduce total execution time significantly but might incur higher costs due to the need for more resources.
Cost for parallelization of an individual test
-
$
+
Major vendors charge ~$130/test/mo for up to 25 parallel runs and ~$100/test/mo for 26 or more.
Number of tests ran in parallel per run
-
+
Max:
625
Example: If you have 200 tests, you might run 4 batches of 50.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
What it takes
Share
Tests for 80% coverage
625
Time to create
625
tests
156
Business days
Test runs
12,500
/ mo
(
625
tests
running
1x
per day
)
100 test failures
100 test runs
Test failures
938
/ mo
Test maintenance
1,407
hrs / mo
QA team to hire
9 QAEs
,
3 SDETs
test
Team cost
$
1,170,000
/ year
1 QAE(s)
1 SDET(s)
With a team of 25 developers, we estimate you’d need 625 tests to reach 80% coverage. To run those tests 1x per day, while factoring in the cost of infra, it would cost you a grand total of $1,170,000/yr.
Running your tests in parallel
1 tests in parallel (625 batches)
Time spent
12,500
hrs / year
Run cost
$
1,560
/ year
Total cost of end-to-end testing in-house
~$
1,171,560
/ year
Compare with QA Wolf