QA & Testing

AI in Software Testing: Generating Test Cases with LLMs

Home Blog QA & Testing

QA & Testing

AI has changed software testing faster than almost any other part of development. Used well, it removes a lot of repetitive QA work. Used carelessly, it produces impressive-looking tests that check nothing. Here's how we use it in practice.

Where AI helps most

1. Drafting test cases from requirements

Give an LLM a user story or a requirements document and ask for test cases covering positive paths, negative paths, boundaries and edge cases. It produces a strong first draft in seconds, often including cases a tired human would miss, such as leap years, empty uploads or Unicode names.

User story: An applicant can upload up to 5 supporting documents
(PDF or JPG, max 5 MB each) before submitting the application.

Generate test cases as a table: ID, title, preconditions, steps,
expected result, type (positive/negative/boundary).

2. Generating test data

Realistic but fake names, addresses, IDs in the right formats, and tricky inputs (very long strings, special characters, SQL-like text) are quick to generate. That's especially useful when production data can't be used for privacy reasons.

3. Writing automation scripts

LLMs are good at turning a manual test case into a Playwright, Cypress or pytest script, especially when you give them your existing conventions and page objects as examples. They're also helpful for explaining and fixing failing tests.

4. Reviewing coverage

Asking "Here are our test cases for the payment flow. What's missing?" is a surprisingly effective way to find gaps.

5. Exploratory testing with agents

Newer AI agents can drive a browser, explore an application and report anything that looks broken. They don't replace a skilled tester, but they're useful for quick smoke checks of new builds.

Where AI falls short

  • It doesn't know your business rules. It will confidently assume behaviour that is wrong for your domain. Expected results must be checked against real requirements.
  • Weak assertions. Generated tests sometimes only check that a page loads, not that the right thing happened.
  • Brittle selectors if you don't steer it towards role- and label-based locators.
  • Volume isn't value. Five hundred shallow generated tests are worse than fifty well-chosen ones, because they're slow and noisy, and people learn to ignore them.

A workflow that works

  1. Human defines scope: critical flows, risks and business rules.
  2. AI drafts test cases and data.
  3. QA engineer reviews, removes duplicates, fixes expected results and adds domain-specific cases.
  4. AI converts approved cases into automation following your framework's patterns.
  5. Engineer reviews the code like any other pull request, especially the assertions.
  6. Tests run in CI. Failures are triaged by people, with AI helping to summarise logs and traces.

Keep data safe

Don't paste production data, credentials or confidential source code into AI tools that aren't approved by your organisation. Use enterprise offerings with clear data-handling terms, or self-hosted models for sensitive projects.

The result

In our experience, AI doesn't replace testers. It shifts their time from typing to thinking: deciding what's risky, designing meaningful checks and exploring the product. That's where good QA always created its value.

Want to bring AI into your QA process safely? Our QA testing team can help set up the workflow and tooling.