Testing Scripts Like Normal Code

Scripts are the testable part

The scripts inside a skill are ordinary programs, so test them like any code: unit tests for functions, and command-level tests that run the script with sample arguments and check the exit code and output (this is what Claude will see). Cover the unhappy paths: missing arguments, a file that does not exist, bad input, a failing network call. Use fixtures (small saved inputs) so tests are fast and need no network. Make sure error messages tell the user (and the agent) what to do. Run the tests in CI if the skill lives in a shared repository. Testing scripts well means the part of the skill that is deterministic is genuinely reliable, leaving only the model's judgement to evaluate.

A command-level test harness, run

I ran this with plain Python 3 (standard library only). The harness writes a tiny helper script, runs it with three argument sets, and checks both the exit code and the JSON output. The negative input exits with code 2 and still prints parseable JSON. All three cases pass.

import subprocess, sys, tempfile, os, json, textwrap

SCRIPT = textwrap.dedent("""
    import sys, json
    n = int(sys.argv[1])
    print(json.dumps({"n": n, "even": n % 2 == 0}))
    sys.exit(0 if n >= 0 else 2)          # exit code 2 = invalid input
""")
CASES = [(["4"], 0, {"n": 4, "even": True}), (["7"], 0, {"n": 7, "even": False}), (["-1"], 2, {"n": -1, "even": False})]

with tempfile.TemporaryDirectory() as d:
    path = os.path.join(d, "helper.py"); open(path, "w").write(SCRIPT)
    for args, want_code, want_json in CASES:
        r = subprocess.run([sys.executable, path, *args], capture_output=True, text=True)
        ok = r.returncode == want_code and json.loads(r.stdout) == want_json
        print(f"helper.py {args[0]:>3} -> exit {r.returncode}, output {r.stdout.strip()}   {'PASS' if ok else 'FAIL'}")

Output:

helper.py   4 -> exit 0, output {"n": 4, "even": true}   PASS
helper.py   7 -> exit 0, output {"n": 7, "even": false}   PASS
helper.py  -1 -> exit 2, output {"n": -1, "even": false}   PASS

Use fixtures, not the network

Saved sample inputs keep script tests fast and repeatable.

Quick check: What should a command-level script test check?

• The colour of the output
• Only that the file exists
• The exit code and the output the agent will see, including error cases
• Nothing; scripts need no tests