# Generating Tests That Mean Something — AI Coding Assistants: Use Them Well and Review Them Hard

Source: https://www.geekswithgeeks.com/en/ai-coding-assistants/td-generating-tests

> Check that generated tests would actually fail if the code were wrong.

## Vacuous tests pass for any code

Assistants can generate many tests fast, but some assert almost nothing: they only check that a function returns *something*, or they copy the implementation's own logic. A quick check is to **break the code on purpose** and see whether the tests fail. A test that stays green on broken code adds false confidence. Also look for missing cases: empty input, boundaries, invalid input and error paths.

## Prove it, then explain it

Assistants write tests and docs quickly, but you must make sure they check something real.

![Three steps: generate, break on purpose, document.](assets/figures/ai-coding-assistants/section-4-map.svg) — Figure 4.1 — Generate, break on purpose and document.

## A weak test vs a strong test

I ran this. Against the correct function both pass. After `is_even` is broken to always return True, the weak test still passes and only the strong test fails.

```python
def is_even(n): return n % 2 == 0
def weak_test():   assert is_even(2) in (True, False)      # always passes
def strong_test(): assert is_even(2) is True and is_even(3) is False

def broken_is_even(n): return True
is_even = broken_is_even
for name, fn in (("weak", weak_test), ("strong", strong_test)):
    try: fn(); print(name, "passes on BROKEN code")
    except AssertionError: print(name, "fails on broken code")
```

Output:

```
weak passes on BROKEN code
strong fails on broken code
```

## Mutation by hand

Change a `>` to `>=`, flip a condition or return a constant, then run the tests. If nothing fails, the tests are not protecting that line. Tools called mutation testers automate this idea.

**Quiz:** How can you check that a generated test is meaningful?

- [ ] Count how many lines it has
- [x] Break the code on purpose and see if it fails
- [ ] Check that it has a long name
- [ ] Delete it

*Answer:* Break the code on purpose and see if it fails. A meaningful test fails when the behaviour it protects is broken.