Reading and Searching a Codebase

Find the right files first

A real repository is far larger than a context window, so an agent must find the relevant code before it can change it. Typical tools: file listing and glob (by name), text search (grep or ripgrep, for symbols and strings), read file (often by line range to save tokens), and a repo map: a compact outline of files and their classes and functions, built by parsing the code, that lets the model see the project's shape in a few hundred tokens and decide what to open. Some tools add semantic search with embeddings, or language-server features such as "go to definition" and "find references". Good agents read selectively and iteratively: search, open a small range, follow a reference, then edit. Reading too little causes wrong edits; reading everything wastes context and money.

See, edit, run, record

A small set of reliable tools — search, read, edit, run and git — does most of the work.

Building a repo map with ast, run

I ran this with plain Python 3 (standard library only), using a throwaway project created in a temporary folder. The tiny sample project used in several examples is created by the make_project helper shown in the first example that uses it; later examples start with the same helper. The script parses every Python file and prints classes, methods and functions: eight lines that let a model see the whole project's shape. The sample project has a small pricing module, a cart and tests.

import os, tempfile, textwrap

def make_project(root):
    files = {
        "shop/__init__.py": "",
        "shop/pricing.py": textwrap.dedent("""
            def apply_discount(price, percent):
                \"\"\"Return price after a percentage discount.\"\"\"
                return price - price * percent / 10

            def add_tax(price, rate=0.18):
                return round(price * (1 + rate), 2)
        """),
        "shop/cart.py": textwrap.dedent("""
            from shop.pricing import apply_discount, add_tax

            class Cart:
                def __init__(self):
                    self.items = []

                def add(self, name, price):
                    self.items.append((name, price))

                def total(self, discount_percent=0):
                    subtotal = sum(p for _, p in self.items)
                    return add_tax(apply_discount(subtotal, discount_percent))
        """),
        "tests/__init__.py": "",
        "tests/test_pricing.py": textwrap.dedent("""
            import unittest
            from shop.pricing import apply_discount, add_tax

            class PricingTests(unittest.TestCase):
                def test_discount_ten_percent(self):
                    self.assertEqual(apply_discount(200, 10), 180)

                def test_discount_zero(self):
                    self.assertEqual(apply_discount(200, 0), 200)

                def test_tax(self):
                    self.assertEqual(add_tax(100), 118.0)
        """),
    }
    for rel, text in files.items():
        path = os.path.join(root, rel)
        os.makedirs(os.path.dirname(path), exist_ok=True)
        with open(path, "w") as f:
            f.write(text.lstrip("\n"))

import ast

def outline(root):
    lines = []
    for dirpath, _, names in sorted(os.walk(root)):
        for name in sorted(names):
            if not name.endswith(".py"): continue
            rel = os.path.relpath(os.path.join(dirpath, name), root)
            tree = ast.parse(open(os.path.join(dirpath, name)).read())
            symbols = []
            for node in tree.body:
                if isinstance(node, ast.FunctionDef):
                    symbols.append(f"def {node.name}({', '.join(a.arg for a in node.args.args)})")
                elif isinstance(node, ast.ClassDef):
                    methods = [n.name for n in node.body if isinstance(n, ast.FunctionDef)]
                    symbols.append(f"class {node.name}: {', '.join(methods)}")
            if symbols:
                lines.append(rel + "\n  " + "\n  ".join(symbols))
    return "\n".join(lines)

with tempfile.TemporaryDirectory() as root:
    make_project(root)
    print(outline(root))

Output:

shop/cart.py
  class Cart: __init__, add, total
shop/pricing.py
  def apply_discount(price, percent)
  def add_tax(price, rate)
tests/test_pricing.py
  class PricingTests: test_discount_ten_percent, test_discount_zero, test_tax

Quick check: What is a repo map for?

• Encrypting the repository
• Showing the project's shape compactly so the model can choose what to open
• Running the tests
• Deleting dead code