As AI gets faster at building and testing software, are we getting better at delivering the right thing, or simply better at checking our own assumptions? Digivante CEO Conor Whelan explores why passing tests don’t always mean a good customer experience, and why questioning requirements is becoming an increasingly important part of QA.
You can build exactly what was asked for, test it thoroughly and still end up with something customers don't want.
That's always been possible in software development. But with AI now involved in writing requirements, building features and generating tests, I think we're creating the potential to do it a lot faster.
And that's quite an interesting problem.
We're making huge progress in how quickly software can be produced and checked. AI software testing tools can generate scenarios, create automation scripts and help teams get feedback in a fraction of the time it used to take.
I'm all for that. If we can use AI to do something reliably in five minutes rather than having somebody spend half a day on it, why wouldn't we?
But I keep coming back to one question.
A perfectly functioning bad decision
Imagine an ecommerce business decides customers must create an account before they can complete a purchase.
The requirement gets documented, the feature gets built and tests are generated to check the process. Registration works, the account is created, the customer can complete checkout and the order is recorded correctly.
Everything passes.
From the perspective of the agreed requirement, that's a successful result.
Mandatory account creation at checkout
Illustrative scenarioRequirementCustomers must create an account before they can complete a purchase.
Automated checks in scope 4 of 4 passed
- Registration works Passed
- Account is created Passed
- Customer can complete checkout Passed
- Order is recorded correctly Passed
The question for the requirementWhy do customers have to create an account?The checks confirm expected behaviour in the scenarios covered. They can't tell you whether mandatory registration is the right decision for customers.
But has anybody questioned why customers have to create an account? Is it necessary? Does it make the experience better for them? Or have we just introduced another step between finding a product and buying it?
There might be a perfectly reasonable business case for mandatory registration. The point is that we'd need to examine that decision, not just confirm that the software implements it correctly.
You could have hundreds of passing tests and still know very little about whether you've made the right decision for the customer.
That's not a failure of the tests. They're doing what they were designed to do.
It's a failure to ask the question in the first place.
When the software factory starts checking its own work
I recently watched a talk by Zach Lloyd, founder of Warp, about software engineering becoming more like factory engineering.
He talks about moving towards a model where AI agents take on more of the work involved in producing software, from specifications and implementation through to review and verification, while people remain involved at important decision points.
I can see a lot of value in that. Development teams are already using AI to accelerate delivery, and I think we'll see the same thing happen across more of the testing process.
What interested me, though, was his emphasis on human taste, input, product sense and guidance at the points where they matter.
Because the danger with an increasingly automated process isn't necessarily that the individual tools don't work.
It's that they might work extremely well, while all operating from the same set of assumptions.
If AI helps write the specification, another system builds against it and another generates tests from it, you could end up with a very convincing set of results.
But those results may all be confirming the same original understanding of what was needed.
How one assumption can pass through every stage
Each stage checks its work against the stage before it. Unless challenge is built in, nothing in this chain asks whether the requirement was right.
- Stage 1RequirementThe original understanding of what's needed
- Stage 2SpecificationWritten from the requirement
- Stage 3BuildBuilt against the specification
- Stage 4Automated testsGenerated from the specification
- ResultAll checks passedConfirms the original understanding
- The same assumption, carried forward unchallenged
- Verification against the specification
Where independent challenge comes in
At stage 1, before the build begins, the requirement is questioned using different context: customer research, analytics, or an instruction to question the design. That challenge can come from a person or an AI tool. Either way, it has to be designed into the process.
AI can absolutely help challenge requirements too. Give it different context, customer research, analytics or an instruction to question the design, and it may identify things the development team hasn't considered.
But that independent challenge needs to be part of the process. It doesn't automatically happen because we've introduced another AI tool.
More automated checks don't necessarily mean more independent scrutiny.
And I think that distinction is going to become increasingly important.
Are we confusing test coverage with confidence?
We've spent years in QA talking about coverage, test execution, automation rates and the number of defects found.
They're useful things to measure. We measure them at Digivante because they help us understand what testing has been done and what it's uncovered.
But they're not the whole picture.
I'd argue there are three different questions we need to answer on a software project.
Three questions, each needing a different kind of evidence. Automation, AI tools, testers and real users can contribute to more than one.
Have we built it correctly?
Typical evidence: checks against the specificationDoes the feature meet the agreed specification? Does it behave as expected? Do the integrations work? Does regression testing show that we've broken something elsewhere?
This is where automation, AI-generated tests and structured functional testing can bring enormous value.
Have we built the right thing?
Typical evidence: challenged assumptionsDid we understand the requirement properly? Does it solve the customer's problem? Have we challenged the assumptions behind it? Is there a better or simpler way of achieving the same result?
Those questions need product understanding and judgement, ideally before development is well underway.
Does it work properly for the people who will actually use it?
Typical evidence: real customers, real contextThis is where the context can change everything.
A checkout might perform exactly as expected in a controlled test but behave differently when somebody uses an older phone, an unfamiliar address format, a local payment method or a combination of promotions that nobody on the internal team considered.
Or it might work technically but still be confusing, frustrating or unnecessarily difficult to complete.
Automated tools can uncover many of these problems. Experienced testers and real users bring another source of evidence, particularly when they're approaching the experience without the assumptions of the people who designed it.
For AI-powered features and AI-built journeys, this is also why real-user evaluation of AI-powered experiences matters. A generated response might be technically valid but irrelevant to the shopper, or an AI-assisted journey might work without being particularly intuitive. You need to see how people actually interact with it.
That broader perspective is what makes testing real customer journeys valuable alongside scripted checks.
We need all three questions answered, not just a bigger collection of evidence against the first one.
The better we get at using AI, the less we might question it
There's another reason I think this deserves attention.
In July, we surveyed 43 members of Digivante's testing community about their use of AI at work.
Of those who responded, 95% said they'd encountered incorrect or misleading AI output in the previous month. Yet only 35% said they always verified AI output before using it.
What really caught my attention was the group reporting the greatest time savings. Among those 12 respondents, only one said they always verified the output.
AI Pulse: what our testing community told us
Survey of 43 members of Digivante's testing community, July. Self-reported responses.
reported encountering incorrect or misleading AI output in the previous month
said they always verified AI output before using it
Among the 12 respondents reporting the greatest time savings, one said they always verified output
It's a small, self-reported survey, and I wouldn't claim it tells us how every development team behaves. It also wasn't a study of software release quality.
But it raises a question worth considering.
If the whole reason we're adopting AI is to save time, does checking the output eventually start to feel like something that's slowing us down?
That's particularly relevant when we think about how software delivery is changing. We could produce more features, more code and more test results in the same amount of time.
But unless we change how we assess the output, we won't necessarily have any greater confidence that we're delivering the right thing.
We explored that behaviour in more detail in our AI Pulse research into the verification gap.
For me, the lesson isn't that we should stop using AI. It's that we need to be deliberate about when and how we verify what it produces.
The most useful QA person might be the one who questions the requirement
I've talked before about how businesses often start the QA conversation in the wrong place.
A company has a replatform to deliver, a new checkout to launch or an internal team that's stretched, and the conversation immediately becomes about which types of testing they need.
I'd rather start with the project itself.
What are we trying to achieve? What's already been decided? Where are the risks? What does the existing team have covered? Is there anything about the requirements that doesn't make sense?
An experienced QA person should be able to get involved at that stage, speak to product and development and question things before everyone is committed to building them.
Sometimes they'll identify an ambiguous requirement. Sometimes they'll spot a missing customer journey or a dependency that hasn't been considered. Sometimes they'll challenge whether the proposed solution is actually the right one.
Those conversations aren't always comfortable, particularly when there's a deadline attached to the project.
But they're considerably easier to have before the work is built than after it's been released.
And this is why I don't think the future of QA should be about protecting the manual execution of tests.
If AI can reliably do some of that work, use it. The value of an experienced QA person is increasingly in knowing what needs testing, understanding the risk, getting the right people and tools involved and making a judgement about whether the results are good enough.
That includes recognising when a set of passing tests isn't telling you everything you need to know.
Someone still needs to own the quality
There's another problem I see fairly regularly.
Product understands one part of the project. Development understands another. There are automated tests running, perhaps external testers involved and now AI tools producing additional output.
Everyone is contributing to quality, which is exactly what you want.
But who's looking at the overall picture?
Who's making sure the requirements have been challenged, the important customer journeys are covered and the findings are being assessed against the actual business risk?
And who has the authority to say that something needs another look, even when the test report is green?
Everyone contributes to quality
- Product
- Development
- Automated tests
- External testers
- AI tools
Who owns the overall QA?
Connecting the work. Understanding the risk. Challenging assumptions.
That accountability can sit inside the team or be supported externally.
Shared responsibility for quality is a good thing. Unclear accountability isn't.
At Digivante, that's why we increasingly look at QA as something to manage around the needs of the project, rather than simply a list of testing services to deliver.
Sometimes a business needs additional testers. Sometimes it needs specialist coverage, real users across multiple markets or support with automation. Sometimes the biggest gap is an experienced person who can work alongside the existing team and take ownership of the QA approach.
Our QA team augmentation offering is one way we provide that expertise when an internal team needs additional support.
The important thing is having somebody who understands what needs to happen and can bring those capabilities together.
Faster software needs better questions
I don't think the answer to any of this is to slow development down.
AI is going to make software faster to produce, and that's a good thing. We should make the most of it.
But if we're saving hours generating code and tests, I'd like to think some of that time can go into understanding the customer, challenging the brief and looking at the experience from outside the development team's perspective.
Because I don't particularly want to see a future where every requirement gets implemented brilliantly, every test passes and we only discover afterwards that customers don't like what we've built.
You can have a very efficient process for delivering the wrong thing.
And as software gets faster to build, I think the people prepared to question what we're building will become more important, not less.
So before we celebrate another release with everything showing green, I'd ask one more question.
Did we prove the software works as expected, or did we establish that we were building the right thing?
They're not necessarily the same thing.
This article was inspired by Zach Lloyd's talk, Software Engineering Is Becoming Factory Engineering, and his discussion of the human judgement needed in increasingly automated software development.
