Metronome provides a usage-based billing platform that helps software companies launch and iterate pricing faster. Their products allow businesses to quickly adjust pricing models, integrate with existing tools, and track usage in real time, making billing a flexible part of product development instead of a bottleneck.
When Metronome first started, the setup was simple with just 14 engineers. Each team member handled their own testing, which was manageable at that scale. But as the company grew, adding more engineers and features, things became complicated.
As more engineers came in, as the product got more complex, then we started seeing some regressions.
—Elia Karagiannis, Product Platform Engineer
The straightforward methods of the early days struggled to keep pace with the growing complexity. New features started interacting in unexpected ways, causing bugs and glitches. The manual testing approach that worked for a smaller team couldn’t catch up with the demands of a scaling operation, leading to a higher frequency of annoying regressions.
The outdated manual testing process wasn’t just inefficient; it became a stumbling block to growth. With each engineer having their own testing method, there was no unified strategy, making it challenging to manage and maintain quality as the platform and its features became more complex.
We were seeing regressions happening…our head of engineering was tired of them.
—Elia Karagiannis, Product Platform Engineer
As the team began planning a new major version of their platform, it became clear a new testing approach was necessary. The manual methods were not only cumbersome but were also inhibiting growth. This realization led Metronome to search for a solution that could scale with their needs and ensure smoother operations without compromising on quality.
As the company moved up market to enterprise customers, they needed to rebuild their front-end and much of their back-end systems. Having QA Wolf’s regression tests constantly checking for problems gave Metronome’s engineers and leadership the confidence to move whole systems and shortened time to market.
QA Wolf was instrumental in the migration from Metronome 1.0 to 2.0. We were actually running parallel environments, old and new. And then when we flipped the switch, QA Wolf updated all the tests over the weekend so we could continue rolling on Monday.
—Elia Karagiannis, Product Platform Engineer
Regressions are stopped before they go out, and requesting test coverage from QA Wolf has been added to Metronome’s definition of done.
We doubled-down on QA Wolf and are now 100% bought into their testing approach, to the point where it's part of our definition of done process. When developers create a new feature they make a quick video that QA Wolf uses to write tests and add them to our test suite.
—Elia Karagiannis, Product Platform Engineer
QA Wolf is now trusted critical infrastructure in Metronome’s engineering process. And the trust that engenders with our clients is probably the most important thing. When you’re dealing with billing it’s vital that there not be any bugs — QA Wolf helps Metronome serve their clients quickly without losing sight of quality.
We have about 400 tests that run in about 11 minutes, and the flake score is under 1%. We rarely get a false negative failure in QA Wolf, and I was actually quite surprised at how responsive the QA Wolf team was on those tests.
—Elia Karagiannis, Product Platform Engineer