ENGINEERING
We deleted 40% of our test suite and shipped faster
Coverage went down. Confidence went up. Here is how we decided what to cut.
Tom Becker
Engineering
Most of those tests tested the framework
When we audited what was actually failing, a clear pattern emerged. Tests that broke on real bugs clustered in a small number of files. Tests that broke on refactors — and only on refactors — made up most of the suite. They were not protecting the product. They were taxing every change.
We kept everything that touched money, permissions, or data integrity. We deleted assertions that restated the implementation line by line. The suite went from nineteen minutes to six.
Coverage is an input, not a goal
A high coverage number tells you lines were executed, not that behaviour was verified. We now track how often a test catches a real defect before release. It is a harder metric to collect and a far more honest one.
