Testing with AI: Turn Requirements Into Reviewed Tests

AI can write test cases from requirements, spot coverage gaps and explain failing tests in plain language. The catch: a test an AI proposes is only as good as the expected outcome you review and confirm. Use AI for the drafting; keep the decision about what counts as correct with you.

Tools like fenbs help by keeping test tasks, review lessons and attributed changes together across sessions, so the next AI session starts with the same context the last one ended with.

What AI does well in testing

AI assistants are good at generating repetitive test cases from a specification, naming edge cases you might overlook and turning an error stack trace into a readable explanation. These are time-consuming steps that do not require deep judgment about the product.

• Generating unit test stubs from function signatures and docstrings.
• Listing boundary values, null inputs and error conditions for a given function.
• Translating a failure message into a plain-language summary of what broke.
• Suggesting additional assertions when a test only checks one outcome.
• Spotting untested branches when you paste a function alongside its existing tests.

AI does not know what your system is supposed to do from first principles. It infers from code and comments. If the code is wrong, AI-written tests often match the wrong behavior rather than the intended one. Your job is to supply the expected outcome separately from the implementation.

Write requirements first, then tests

Start with a requirement or acceptance criterion in plain language, not a function to test. That order forces AI to check whether the code matches the requirement rather than whether the tests match the code. A sentence a non-developer can read aloud is usually precise enough.

1. Write what the function or feature should do in one sentence, before touching the code.
2. Ask the AI to generate test cases from that requirement, not from the existing implementation.
3. Review each proposed test case against the requirement yourself before you run it.
4. Run the tests. For any failure, paste the error into a new session and ask for reproduction steps.
5. Record what each session verified and what it skipped in a task note, not just a passing count.

Checking coverage without trusting AI's description

AI can describe coverage in words, but those descriptions are often optimistic. Use your test runner's coverage tool, such as Istanbul for JavaScript or pytest-cov for Python, to get actual numbers. Then paste the uncovered lines into the AI and ask which cases are worth adding. You decide whether to add them.

For AI bot testing, coverage means something different: you need to check conversation flows and edge inputs, not just code branches. Write the expected conversation behavior first, then generate and review test cases in the same way.

Recording what was tested

A test that passed in one session is not evidence of correctness in the next session if the code changed. Record what was verified, not just that tests passed. A note like "verified null input throws TypeError with message X" is more useful than "tests pass."

When you find that a lesson applies across sessions, for example "always seed test data before checking sort order," write it down as a Lessons learned entry in fenbs. The next AI session reads it before touching the same area.

When an AI fix breaks a passing test

An AI asked to fix a failing test will sometimes adjust the code to make that one test pass, without realizing another test depended on the previous behavior. Always run the full suite after an AI-suggested change, not just the test that was failing.

When a fix breaks something unexpected, record the reproduction steps and what the passing test expected. That gives the next session enough context to propose a repair that satisfies both requirements. History in fenbs records which session made each change, so you can trace back to when a test first broke.

What AI cannot check for you

AI cannot verify that your tests cover the right behavior if it does not know what the right behavior is. It cannot tell you whether a passing test suite means the product is correct, only whether the code matches the tests. And it cannot catch a requirement that was left out entirely: a test that does not exist will never fail.

These are the gaps a developer fills. Review the test list against the requirements, not just against the code. Ask whether there are important behaviors not covered by any test. AI makes writing the tests faster; the judgment about what needs to be tested stays with you.

How to do this with fenbs

1. Create a free account and add your testing project.
2. Add a task for each testable requirement. Write the requirement in the task note in plain language.
3. After each AI session, update the task with what was verified, what failed and any edge cases the AI suggested.
4. Add a Lessons learned entry when a session reveals a recurring pattern the next session should know.
5. Connect your AI app over MCP from Integrations and ask it to read the project context, including open tasks and recent lessons, before proposing new tests.