E2E tests: always a priority, never in the sprint

It is the last day of the sprint. The feature works: the engineer demoed it on the call, the PM clicked through it on staging, QA gave it a pass. It happens to touch checkout, which is one of the few flows in the product that everyone agrees must never break. The ticket has one unchecked box left, "add E2E coverage for the new checkout path." Someone types the familiar sentence into the thread: let's fast-follow the E2E test. Nobody objects, and there is not much to object to: the feature works and the sprint is over. The ticket closes, the release goes out, and the test joins a backlog column it will never leave.

TL;DR E2E tests get deferred because the incentive to create one and the cost of skipping it are separated in time and in ownership, so every individual deferral is locally correct. The debt then compounds two ways: a test nobody created leaves no artifact, so the gap is invisible, and the tests that do exist break on routine UI change. Mandates, QA backfill, hardening sprints, and AI generation all push against that gradient rather than change it. You do not need E2E coverage everywhere, only on the journeys where being wrong is expensive and irreversible, and there you need to trust it. If you do one thing this week, have the people who would be paged list independently the three journeys whose failure would be an incident, compare the lists, then check which had a test that passed against this week's build.

Ask any team whether end-to-end tests matter and you get an unqualified yes. Ask when they last added one and the answer gets quieter. E2E coverage is the rare thing that is simultaneously a stated priority and a permanent backlog item.

That gap has a shape worth naming. At every decision point, skipping the test is slightly easier than creating it, so the system slides downhill one reasonable decision at a time. Call it the deferral gradient. No one decides the critical journeys should go unwatched. The gradient decides.

The deferral is rational in the moment

A feature ships under a deadline. The unit tests come along nearly for free, because the engineer is already in the code and the feedback loop is measured in seconds. The end-to-end test is a different category of work: it needs a running app, realistic data, a stable selector strategy, and a mental model of what a real user does across several screens. None of that is in front of the engineer at the moment they would write it, unless the team has already paid to put it there.

So it gets deferred, not because anyone undervalues it, but because every individual deferral is locally correct. The feature works, QA looked at it, and the ticket needs to close. The E2E test is the one piece of the definition of done with no immediate forcing function, and skipping it costs nothing today.

That is the core of the problem. The incentive to create the test and the cost of not creating it are separated in time and often in ownership. Any system with that shape accumulates debt.

Why the debt compounds instead of staying flat

Missing tests would be tolerable if the cost stayed constant. It does not. Two mechanisms drive it up.

The gap is invisible, so it only grows

Code coverage is easy to measure. A missing E2E test is not, because no artifact exists whose absence you could see, and no tool reports the user journey nobody tested. Nobody opens a pull request saying "this checkout path has no end-to-end coverage." So the gap is never prioritized, and it widens every time a change lands on a journey nobody is watching. Most teams learn its true size during an incident review, counting backwards from a production failure to the test that was never created.

The tests you do have rot

The suite a team does manage to produce becomes a liability the moment the UI changes, and the life cycle is always the same. A checkout flow gets a test. Months later a routine refactor renames a component and the selector no longer matches. The suite goes red, someone burns an afternoon confirming nothing is actually broken, and patches the selector by hand. Next refactor, red again, and this time it is skipped "temporarily." The gap that leaves is silent, per the mechanism above, and checkout is now unwatched until a real regression walks through it.

This is not a rare failure mode, and it falls hardest on exactly the tests we are talking about. When Google measured flakiness by test size, 0.5% of its small tests were flaky in a given week, against 14% of its large ones, where large means the broad-scope tests that touch a running system. Nor has modern tooling fixed it: across 30 web applications tracked over five years of releases, roughly a quarter of element locators broke from one release to the next.

Engineers draw the obvious lesson from a suite that is usually red for reasons unrelated to any defect: ignore it. At that point it has negative value, costing maintenance time and providing no signal, and the flaky tests get quarantined.

A quarantined test is a deleted test with extra steps.

What the compounding actually costs

The metric that matters to whoever owns delivery is not line coverage. It is whether the team can release without someone volunteering to watch the dashboards for an hour afterwards. When coverage is thin and the suite is untrusted, every release becomes a judgment call on incomplete information. Velocity does not drop because people slow down. It drops because the uncertainty tax gets paid in hesitation, manual verification, and rollback. This is the industry's default state, not an edge case. Asked what most often delays a release, 47% of the developers GitLab surveyed pointed at testing. Not at testing too much: the reasons they gave were not having enough tests and not having automated the ones they had, with only 12% reporting full test automation.

Hesitation is hard to put a number on, which is part of why this cost goes unargued. The other half of the bill is not hard to size at all, and you do not need anyone else's model to do it. Measure how many hours your team spent last quarter repairing tests that had not found a defect. That number is usually available, rarely looked at, and considerably more persuasive in a budget conversation than any published figure.

When thin E2E coverage is the right call

There is a serious counterargument here, and it deserves better than a strawman.

Plenty of good teams deliberately keep E2E coverage thin and invest in detection and recovery instead: feature flags, progressive rollout, real user monitoring, a rollback that takes ninety seconds. If a bad release reaches 1% of traffic, trips an alert, and is reverted before most users notice, recovery bought you more than prevention would have. For a large class of web products that is the correct allocation, and a team that made that call deliberately is not the team this post is about.

It stops working where recovery is not enough. Some actions cannot be rolled back: money moves, data is deleted, an email goes out, a compliance boundary is crossed. Detection also assumes someone is watching the right signal, and the failures that hurt most are silent. A checkout that succeeds for everyone except users on one payment method does not trip an alert. It arrives as a support ticket three weeks later, with the revenue already gone.

So the honest version is narrower than "you need E2E coverage." You need it where being wrong is expensive and irreversible, and you need to trust it there. For everything else, ship behind a flag and watch the graphs. That narrowing is good news: the target is a handful of journeys, not a coverage percentage.

Why "just write more tests" does not work

The standard responses are familiar, individually reasonable, and all well tried.

• Mandate coverage in the definition of done. This works until the first hard deadline, at which point the mandate is the first thing negotiated away, because the person under the deadline is not the person who pays for the gap.
• Hire QA to backfill. This converts an engineering bottleneck into a QA bottleneck plus a coordination cost. The tests still rot when the UI changes; now a different team owns the rot.
• Dedicate a hardening sprint. This produces a coverage spike that decays on the same curve as before, because nothing about the underlying incentive structure changed.

None of these are wrong. They are just fighting the gradient instead of changing it. The deferral keeps happening because the work is expensive at exactly the moment it has the least urgency, and the artifact, once created, is brittle against exactly the changes teams make most often.

The answer you are already skeptical about

There is a newer response: generate the tests with AI. The skepticism that idea attracts is earned, so it is worth being specific about what is actually wrong with it.

A model that has never seen your application will produce tests that read well and assert nothing. Handing it the repository helps less than it sounds. It can now see what the code does, but assertions derived from an implementation that already exists can only confirm that the implementation does what it does. If the discount was meant to apply at ten items and the code says eleven, a test generated from that code faithfully encodes the bug as expected behavior, and does so more convincingly than a guess would. Source tells you what the system is, not what it is for, and most real defects live in the gap between those two.

More to the point, generation on its own touches neither mechanism above. A test that exists but has never earned anyone's trust is still an invisible gap, because nobody blocks a release on it. A generated test pinned to a fragile selector rots on the same schedule as a handwritten one, arguably faster, because no human ever reasoned about why that selector was chosen.

What changing the gradient actually requires

If the problem is structural, the fix has to be structural too. A durable solution has to change at least three things, regardless of how it is built.

1. Decouple the cost of coverage from the deadline. As long as writing the test competes with shipping the feature for the same engineer-hour under the same clock, the test loses. The work of producing coverage has to move off the critical path of the person who is shipping.
2. Survive UI change without manual repair. A suite that goes red on a component rename is not protecting you from regressions, it is manufacturing false ones, and coverage that cannot absorb routine refactors gets ignored. Ignored coverage is worse than none, because it still costs money.
3. Make the gap visible. Missing coverage on a critical path should be as legible as a failing build, not something a team discovers during an incident review.

None of the three is free. If something else produces the tests, somebody still has to decide whether they describe the behavior you actually want. Absorbing UI change means matching on more than a brittle string, which trades loud false failures for a smaller class of quiet mismatch. Observation has a floor here too: a feature broken badly enough never to render is a feature nothing can watch you lose.

What to do about it on Monday

You can locate yourself on the gradient in an hour. Get the people who would be paged to write down, independently, the three journeys whose failure would constitute an incident. Then compare:

1. Did the lists match? Teams assume this is the easy part and routinely find they disagree. The disagreement is the finding, because you cannot have deliberate coverage of a set nobody has agreed on.
2. Which of those journeys had a test that ran, and passed, against this week's build? This is usually where the room goes quiet.
3. When the suite goes red, what is the team's first assumption: real defect, or flake? If the honest answer is flake, the suite has already stopped protecting you.

Merge the lists into one, put a named owner on each journey, and add one honest column: what, if anything, automatically exercises this today. It fits on an index card.

The third column is the one that stings. That list is what the gradient depends on you not having, and it satisfies the visibility requirement without buying anything. It also beats a coverage percentage as a definition of done: every journey on it has coverage that ran against the current build this week, and when that coverage goes red the team's first assumption is that something is broken rather than that the suite is lying. That measures trust rather than volume.

Where this leaves you

Most teams do not end up with an empty E2E suite. They end up with a thin one carrying a layer of dead tests: journeys where coverage was never created because the deadline won, and journeys where it was created once and has quietly stopped being trusted. Those get treated as different problems. They are two outputs of the same system, and the teams that escape it are not the ones with more discipline. They are the ones that moved the cost off the deadline and made the gap impossible to not see.

There is also reason to think the gradient is steepening. Generating a feature got dramatically cheaper over the past two years. Verifying the feature actually works end-to-end did not, which widens the exact asymmetry this post is about. The informal backstop is thinning too: a suite has always been partly a substitute for the author's own understanding of what they changed, and that understanding is shallower when the change was generated, skimmed, and merged. More code, landing faster, with less human context behind each line, is precisely the condition under which critical journeys need verifying by something other than the assumption that they still work.

These are some of the challenges we work on at detektra.ai. We are not going to claim they are all solved, but we can say that the three requirements in this post are the ones our work is organized around. If the gradient in this post looks familiar, we would like to hear how it shows up on your team.

References

• Jeff Listfield, Where do our flaky tests come from?, Google Testing Blog (2017). The source of the size breakdown: 0.5% of small tests flaky in a week against 14% of large ones. Note that Google's size labels are assigned by the engineer writing the test, not derived from a strict definition, so read them as scope rather than as a precise taxonomy.
• John Micco, Flaky Tests at Google and How We Mitigate Them, Google Testing Blog (2016). The companion piece, with the widely quoted "almost 16% of our tests have some level of flakiness" figure. That number covers the whole corpus at every size, not end-to-end tests specifically.
• GitLab, What blocks faster code release, Global DevSecOps Survey. Asked what most often delays their code, 47% of respondents named testing, ahead of planning at 39% and code review at 28%. Note what the answer does and does not say: testing is the stage where releases get stuck, and the reasons respondents gave were insufficient tests and insufficient automation, with 12% reporting full test automation. It is not a finding that testing is wasteful.
• Anton Kluge and Andrea Stocco, Web Element Relocalization in Evolving Web Applications, Empirical Software Engineering (2025). Tracks element locators across 30 web applications and 16 versions each, September 2018 to September 2023. The source of the locator-breakage rate above. It also measures what repair costs per release, 0.60h when tests use ID-based locators against 3.05h for XPath, and extrapolates that into annual figures for a hypothetical organization. The repair times are the measured part and travel better than the extrapolation does.
• Agile Alliance, Technical Debt is a Systemic Problem. The economics under the deferral gradient are not new. The half about cost arriving later is temporal discounting; the half about cost landing on someone else is a tragedy of the commons, which is how technical debt gets described when people study it seriously. This piece makes that second case, framing debt accumulation as a commons problem rather than a discipline problem, which is the same argument this post makes about coverage. On a team it does not feel like either one. It just feels like the test never being this sprint's problem.
• Adam Tornhill and Markus Borg, Code Red: The Business Impact of Code Quality (2022). A quantitative study of 39 proprietary codebases: low-quality code carried 15x more defects and took 124% longer to resolve issues in. Useful when you need the compounding cost of deferred quality work in numbers rather than anecdote.