Readable HTML Without JavaScript

What a simple crawler sees

Many AI crawlers fetch the page's HTML and do not execute JavaScript. If your pricing, product details or articles are injected by client-side scripts, those crawlers may see an empty shell. Prefer server-side rendering or static generation for important content, give each page a unique URL, a clear <title>, headings and descriptive text, and avoid hiding key facts in images or only in interactive widgets. A quick test: view the page source, or fetch it with curl, and check that the key text is there.

Raw HTML vs script-injected text, run

I ran this. A text extractor that ignores scripts sees the heading and the paragraph but not the pricing line that JavaScript adds later, so Pricing is not found.

from html.parser import HTMLParser

html_raw = ("<html><body><h1>Acme Tasks</h1><p>Simple boards for small teams.</p>"
            "<div id='app'></div><script>document.getElementById('app')"
            ".innerHTML='<p>Pricing from 5 USD</p>'</script></body></html>")

class Txt(HTMLParser):
    def __init__(s):
        super().__init__(); s.t = []; s.skip = False
    def handle_starttag(s, tag, a): s.skip = tag in ("script", "style")
    def handle_endtag(s, tag): s.skip = False
    def handle_data(s, d):
        if not s.skip and d.strip(): s.t.append(d.strip())

p = Txt(); p.feed(html_raw)
print(p.t, "Pricing" in " ".join(p.t))

Output:

['Acme Tasks', 'Simple boards for small teams.'] False

Check with curl

Run curl -s https://your-site/pricing | grep -i "5 USD". If the number is missing from the response, many crawlers will miss it too, however good the page looks in a browser.

Quick check: Why can client-side-only rendering hurt AI visibility?

• It makes the site too accurate
• JavaScript is banned by browsers
• Crawlers that do not run JavaScript may see an empty page
• It deletes your robots.txt