AI for Writing Unit Tests: Test Behavior and Review Gaps

An AI assistant can generate a full set of unit tests in seconds. Whether they are useful depends on what you ask for. Tests generated from "write tests for this function" tend to cover the happy path and little else. Tests generated from "this function must handle an empty list, a list with duplicates, and a None input without raising" tend to find the bugs you care about.

The discipline of specifying behavior and failure risks before asking for tests pays off twice: the tests are better, and the specification reveals gaps in the function itself. Tools like fenbs keep test requirements and failure lessons in a shared record, so the knowledge from one testing round improves the next. For broader test automation patterns, test automation covers the wider context.

What makes a unit test useful

A useful unit test fails when the function does the wrong thing and passes when it does the right thing. A test that always passes gives false confidence. A test that catches a bug once but cannot be run again without manual setup provides only one-time value. Useful tests are deterministic, fast, and specific about what behavior they verify.

Most AI-generated tests default to testing the structure of the output rather than the behavior. They check that a function returns a list rather than checking that the list contains the right items in the right order under specific conditions. That is a starting point, not a complete test suite.

Specifying from behavior

Before asking for tests, write down the behaviors the function must exhibit. Use plain language: "when the input list is empty, the function returns an empty list" is a test case. "When the input contains a None, the function raises ValueError with the message 'input contains None'" is a test case. When each behavior is one sentence, the assistant can write a test for each one.

Include the full range of valid inputs: typical values, boundary values (zero, maximum, first, last), and invalid inputs. A function that handles strings should be tested with an empty string, a string of maximum length, a string with special characters, and a non-string input.

When you specify behavior this way, the specification doubles as documentation. A future developer reading the test file can see what the function is supposed to do without reading the implementation.

Specifying from failure risks

Failure risks are the things that could go wrong that the happy-path tests will not catch. List the realistic ways the function could receive bad data, encounter an unavailable dependency, or produce incorrect output for a valid input.

• Data quality: None values, empty collections, unexpected types, values out of the expected range.
• External dependencies: a database call that returns no rows, an API call that times out, a file that does not exist.
• Concurrent access: a function called from multiple threads, a shared resource modified during the call.
• Configuration: different behavior when an environment variable is set or unset.

Ask the assistant to generate one test per failure risk. A test for "the database call returns no rows" is more valuable than a second test for the typical case. Failure-risk tests are the ones that actually catch bugs in production code rather than just confirming the function runs.

Reviewing AI-generated tests for gaps

After the tests are generated, read each one and ask whether the assertion would fail for an incorrect implementation. A test that asserts len(result) >= 0 will always pass. A test that asserts result == [3, 1, 4] for a specific input will fail if the function returns the wrong values.

Check the coverage against your behavior and failure risk list. If a behavior you specified does not have a corresponding test, add it. The assistant may have grouped behaviors into fewer tests or omitted edge cases that seemed obvious from the description but were not.

Check the test setup. Mocked dependencies should be reset between tests. Fixtures should represent realistic data rather than trivially correct values. A test that passes because the mock always returns success is not testing the code path where the real dependency fails.

Handling edge cases the assistant missed

When you find an edge case the tests do not cover, add it and note why it was missed. Either the specification did not include it, or the assistant interpreted the specification differently than you intended. Both are useful observations: the first means the specification needs work; the second means the instructions need to be more explicit.

A test you had to add manually is a test you should save as an example for next time. A task description that produces consistently complete tests is better than one that always needs manual additions. Refine it with each round.

Keeping test requirements across sessions

A project's test requirements build up over time. The testing framework and runner. The coverage threshold. The policy on mocking external dependencies. The pattern for testing asynchronous code. Each requirement exists because someone decided it was important. When it is not recorded, the next session may miss it.

These requirements are not just useful for the AI assistant; they are useful for any developer who joins the project. A shared record of what testing should cover and how it should be structured is project documentation that happens to also be AI context.

How to do this with fenbs

1. Create a free account. Add the project with its testing framework and current coverage status.
2. Record test requirements as standing rules in Decisions and rules: which testing framework, which patterns for mocking, what coverage is required, what test types apply to which function categories.
3. When a test reveals a new failure mode, record it as a lesson: what the test found, what the root cause was, and what to specify in future briefs to catch the same pattern earlier.
4. Connect your assistant over MCP from Integrations. Ask it to read the project context, including lessons, before each testing session.
5. Review the test output against the behavior and failure risk list before marking a task as tested. Add missing cases before the session closes.