Why AI Won’t Replace Manual Testing

The world hasn’t been the same since generative AI models. What’s been the same though is people’s fears over losing their jobs to AI.

While the broader impact of AI is yet to be fully determined, QA (Quality Assurance)—an area our team has years of expertise in—won’t experience the doomsday outcomes many predict. Here’s why:

AI Assists People, It Doesn’t Replace Them

Generative AI has integrated into our teams’ lives. While not ubiquitous yet, the team uses AI to aid them in being orders of magnitude more productive when writing emails, updates, test cases, or just writing code for Lidana—our software that integrates AI into the manual testing process.

AI also increases the expectations of quality from the people on the team. Framing of status updates, reasoning about code, poking holes in system design, and so much more are now an expectation across teams that we interact with.

Almost 2 years after Chat GPT was first announced, we are yet to see any reason for humans to be replaced—instead it’s given us plenty of reasons for humans to operate far more effectively and efficiently.

Manual Testing is Still a Collaborative Process

Bystanders often perceive QA as a mundane task that can be handed off to a team and forgotten. This may be true—if you want bad outcomes.

In reality, just like any other profession in software development, manual testing, writing test cases, executing them, keeping your test suite up to date as sprints lead to evolutions, require the testing team to be in the loop.

This means humans…talking to other humans. Slack, Figma, JIRA, email, the format is irrelevant. Unless the humans can communicate with one another, even the most well-trained models won’t be able to derive intent unless a human is able to communicate what they want.

Humans Are Still Responsible

Email summarization makes for a great technical demo, but in reality, the sender still assumes you’ve read the full email and haven’t missed important details.

Living in a world where one might say “I missed that escalation because my AI didn’t put it in the summary” would be an interesting world. Thankfully—in this author’s view—we aren’t in such a world yet.

Similarly, with manual testing, at the end of the day, a test suite with poor coverage, or test cases that just went…er…missing…aren’t going to come down to misbehaving AI. Humans operating that AI remain responsible.

Hallucinations Haunt Us, Still

You can’t make this up (pun intended). AI hallucinations started off as bewilderment and comedy. However, as the technology has matured (or maybe usage, and expectations have adjusted) such occurrences have reduced (at least in pop culture). It’s not down to zero though as the link suggests.

The bottom line is that AI cannot be trusted blindly. With computer programs, outcomes are deterministic. “2 + 2” will always have the same output. With AI, however, we’re dealing with probabilistic systems. “What is two plus two?” has a very high probability of being right. But is it 100%?

That’s why human reviews are imperative, and unavoidable.

—

These and other reasons are what led us to building Lidana, a modern manual testing tool that democratizes manual testing with AI. Check it out here.