My AI agents audited a website. 7 specialists dispatched. 50 pages crawled. Results came back.
The difference between a workflow, and agentic pipeline with a brain.
Every single agent said the same thing: "I can't do my job. There's no content."
The site was a React SPA. Our Spider crawled it with curl. Got back empty HTML shells. Every page: same title, same canonical, 12 words of content. A div with id="root" and nothing else.
The DOM agent couldn't check titles. They were all identical.
The Links agent found zero internal links. The graph was empty.
The Performance agent saw a 4KB shell loading a 2MB JavaScript bundle.
The IL Strategist said "all pages have zero crawlable content."
Every agent correctly identified the root cause: no server-side rendering. But none of them could do their actual job.
So I asked all 7 agents: "Based on this audit, what tool improvements would help you do your job better?"
Spider: "I need Playwright rendering so I can save what the page actually looks like after JavaScript runs."
DOM: "Give me both versions. Raw HTML AND rendered HTML. I want to compare them."
Performance: "I need CrUX API and PageSpeed Insights API keys for real-world performance data."
IL Strategist: "I need a URL validator script and rendered content to find link opportunities."
Links: "The link graph was empty because all links are generated by JavaScript. I literally had nothing to analyze."
The Lead Auditor collected all the feedback and created a backlog. Prioritized by impact. P0: Spider needs JavaScript rendering. Without it, every CSR site audit is useless.
So we planned it. Not a quick hack. A proper engineering spec:
Pre-crawl: sample 10 pages. Fetch each with curl AND Playwright. Compare 6 signals: canonical, title, word count, H1, meta description, paragraph count. If static matches rendered, skip Playwright for the full crawl. If they don't match, trigger full JS rendering.
Three modes: SSR (curl only, fast), CSR (Playwright, thorough), Hybrid (smart mix). The Lead decides which mode based on Spider's pre-check. Every sub-issue gets the mode. Every finding gets tagged: found in static HTML, found after rendering, or found in both.
The agents didn't just find the problem. They told us what to build. They prioritized it. They wrote the requirements. The engineering loop runs itself.
This is what "self-improving agents" actually means. Not marketing fluff. Real agents hitting real walls and creating real engineering tickets to tear those walls down.
Next: build the rendering detector, update the crawler, re-run the audit on the same site. Compare before and after. See what the agents find when they can actually see the content.
I'm going to surf. The agents will plan, build, implement, and test.