TF:2025
My first test:fest talk in 2015 was about testing versus checking. Ten editions later I was on the same stage talking about critical thinking and AI. It was the same question, wearing new clothes.
Most people treat a tenth edition as a celebration. It is that. It is also an audit. When you have stood in front of the same community for ten years, you can check whether the things you said back then still hold.
I never published the recap of test:fest 2025. When I sat down to write it, the history kept getting in the way, so I wrote the history instead and left the edition summary in draft. This is that summary, written up more than a year late. I won't go back over the decade. This post is about one Saturday in Wrocław on 24 May 2025, and about what made that room different from every other room I spoke in that year.
A home crowd is the hardest audience for a new idea
2025 was my biggest speaking year: five conferences in four countries. At least three of them heard some version of the same talk, "Critical Thinking rules in the AI-Enhanced Software Testing". test:fest was the room where it mattered most.
A home crowd usually sounds like the easy room. For a new argument it is the hard one. Abroad, people take you at face value: you are the speaker on the agenda with a title and a slot. At test:fest a lot of the room had watched me argue about exploratory testing on a panel in 2015. They had sat through my metrics workshop and my personal-branding talk. They have a baseline for me. If a new talk is only the old opinions with "AI" pasted on top, a home crowd can tell.
That made it useful. Whatever the talk did in other rooms, this was the one place that could tell me whether it was a new idea or a familiar one dressed up.
Testing versus checking never went away. It got automated.
In 2015 the argument was about testing versus checking. A check confirms what you already expect. A test is an investigation that can surprise you. Back then the question was whether teams were doing too much of the first and calling it the second.
The 2025 talk was the same argument at a different scale. An AI assistant is a very fast generator of plausible checks. It writes assertions that match the code in front of it, and the result looks like rigour. Whether any of those assertions matches what the product is supposed to do is the critical-thinking question, and the tool does not ask it for you.
I see it clearly now, because the year after test:fest I built a production iOS app through AI orchestration and hit exactly that wall. The AI-generated tests had adapted to the code, not to the requirements. Tens of thousands of green checks proved the code agreed with itself. That is checking at industrial volume, and the 2015 distinction explains it better than any 2025 vocabulary does.
So the full circle from my first test:fest talk to the tenth edition was not nostalgia. It was the same failure mode with a faster engine behind it.
The check for your own team: take ten tests your AI tooling wrote last month. For each one, can you name the requirement it protects without looking at the implementation? Count the ones where you can't. That number tells you how much of the suite is checking that looks like testing.
Ten editions earn the right to change shape
The other thing that made 2025 different from the nine editions before it was what came after. As came up on the Testing Station podcast with Marta Firlej, test:fest is moving towards a meetup format in 2026.
Thinking about it since, what I feel is respect. Most events grow until they turn into a trade show with talks in it. Deciding to get smaller, more frequent and more conversational after ten years is a quality decision. Someone looked at what the community actually needs and did not just optimise for the next attendance number. I have written before about measuring for decisions, not for numbers. A conference that changes its format when the metric everyone would expect (more people, bigger venue) points the other way is doing exactly that. I can't think of a better way to close a ten-year run.
What I took away from the tenth edition
Looking back from October 2026, test:fest 2025 was the first draft of an argument I spent the following year proving to myself the hard way. The talk said critical thinking has to stay human while the volume of output goes up. Then came the orchestration work, the review-bias problem, the BowSmith launch and the book chapter. All of it turned out to be that argument with real code behind it.
If you gave a talk years ago and still believe it, that is fine. The better test is whether you can show that the old idea explains something new. Mine did, and the room that showed me was the one that had known me longest.
If test:fest has been part of your path too, I would like to hear which edition was yours. If you are working out where critical thinking fits in an AI-heavy test process, book a Quality Coffee and we can talk it through.
