A few days ago I sat down with a question I have been avoiding for a while:
Are test cases going to be used in the future at all? Will we still need defined test steps? Agents are getting faster and faster. Sooner or later they will just crawl the app, figure out what changed, find the bugs by themselves, and compare the results against notes they took on the previous run.
It is the kind of question that ruins a perfectly good afternoon, because if you run a test automation startup like I do, there is no version of the answer where you get to keep doing what you were doing.
So I went looking for evidence instead of opinions, expecting to find arguments. Instead I ran into a small open-source project called Jev, an agent that walks through an app on its own — no written steps, no recorded clicks — clicking through screens, noticing what changed and where it got stuck. And it is fast, incredible fast!
That was the thing I had been describing as a future product, sitting in a repository. Which made the question much less comfortable, because now it was not a thought experiment. It was a thing people had already built.
I spent a few days on it. Here is what I concluded.
The Short Version
The hand-written test case, with every step spelled out, is on its way out.
Test cases themselves are not. They move up a level: instead of click this button, then that one, then check this label, you write start here, follow this path, and the order total should be 42 euros.
Somebody still has to say what the app is supposed to do. That is the part that does not automate away.
The Crawler Already Exists
The one I stumbled on is jev-browser. Give it a task and a URL, and the agent decides one move at a time, then hands back to plain code that says when to stop.
That is, almost exactly, the shape I was sketching on a whiteboard. Two things about it changed how I think, and neither is about how good the model is.
The decision is autonomous. The test is not. The agent picks each move, and everything around that choice is ordinary code: time limits, what to do when something fails, when to give up. So the "autonomous" part is narrower than it sounds. A crawler can walk an app nobody wrote steps for, which is the part I had to concede. But the route is still chosen by somebody. Today that choice is written down in plain code, and if it ever moves into the model, it has not gone away — it has just become harder to see, which is its own problem.
The model was never doing the hard part. The interesting questions — have we seen this screen before, is this screen worth exploring, does this change matter — are all about what happened. None are about what should have happened. Asked whether any of that behaviour is wrong, the crawler has nothing to work with but the previous run. Getting better at exploring does not move it any closer to knowing what the app was for.
Why Crawling Alone Cannot Tell You If the App Still Works
Crawl the app twice and compare the two crawls, and you get a list of changes. Copy edits that were intentional. A/B tests. Ads. Feeds. Prices. Timestamps. Avatars. Animation frames. A carousel that happened to be on slide three.
Turning that list into a verdict requires knowing what the app is supposed to do, and writing that down is exactly what a test case is. Whether the new behaviour is the correct behaviour is a decision about the product, not something you can read off the screen. The app is not hiding the answer; the answer is not there. And the numbers are unkind: you get more differences with every screen, every scenario, and every run, while the number of people who can review them does not change. A crawler in CI produces noise until somebody mutes it, and muting is how test suites die.
There is a second problem, and it is mechanical. Comparing two runs only works if both runs visited the same screens. If the first run saw screens A, B, and C, and the second saw A, B, and D, then most of your differences come from the run taking a different path — not from the app changing. You get a "new screen" report and a "missing screen" report, and neither of them is a bug. To make the comparison mean anything, you have to pin down where the run goes. And a fixed route through the app is a test case, one level up. The crawler needs the thing it was supposed to replace.
The third problem is reach. A crawler starts at the front door and explores what it can get to. It cannot set up "a user with two items in the cart, an expired card, prices in euros, and an admin role." Those situations are what a written-down test case creates, and they are where the expensive bugs live. Notice what kind of bug that is: the screen loads, the app does not crash, and the bot has nothing to complain about. A total is wrong by four cents. A discount was applied twice. A balance is out of date. Those are only visible against a statement of what the number should have been, which is what a test case is for.
Crawling genuinely finds crashes, dead ends, error states, unreachable features, and broken screens. That is finding problems. It is not proving things still work, and those are two different jobs with two different buyers.
So make it concrete. When a crawler reports a difference, who signs off that it is not a bug? If the answer is "the agent decided," you have built a tester that grades its own homework. If the answer is "a person reviews a short, ranked list with the evidence attached," you have a product.
What Stays Valuable If Crawling Gets Cheap
If crawling becomes cheap and common, value moves to the parts a crawler cannot do: the test case that says what "correct" means, the setup that reaches the situations that matter, the review of what changed, and the record that lets you defend "this changed and that is fine" six months from now.
Which means the crawler should feed the test library, not replace it. Let it explore, let it suggest, let it surface the screens nobody wrote a test for. But the moment it finds something, a person decides what that means, and the decision becomes part of the test case.
That is not a retreat from automation. It is the part of testing that was never about clicking fast in the first place: deciding what the app is supposed to do, and writing that down where a machine can check it.