Why I replaced regex with AI in Siteval's scoring engine.
16 of 60 criteria were originally rule-based. Detecting a live chat widget meant maintaining a list of script names — Intercom, Zendesk, HubSpot, and so on — and running regex against the page source.
Two problems emerged.
1. Anything off the list is invisible forever
My own site,
chartd.net, has a chat icon in the bottom-right corner. It wasn't detected because the script name wasn't in the list. The more patterns you add, the more gaps you create.
2. User feedback goes nowhere
Rule-based checks don't generate AI requests, so good/bad feedback from users had no path to improving accuracy.
Fix: switched 13 criteria to AI scoring. Instead of a pattern list, I just ask: "Is there a live chat widget visible on this page?" Works on anything.
I also wired up a self-improvement loop — Claude (claude-sonnet-4-6) analyzes accumulated ai_requests logs and surfaces weak prompts automatically. AI finding its own blind spots and suggesting fixes.
siteval.chartd.net
#IndieHacker #AI #WebDev #BuildInPublic