Nailed it - this is exactly why I dropped commonly-used UI libraries. It would take 1-2 seconds for things like a dropdown to be usable. Lately, I've been using DaisyUI and Datastar signals, and it has been great.
The average framework-specific component library requires JS file imports, JS props, JS execution on server and JS hydration on browser, JS bundling and JS tree shaking just to give you a piece of HTML and at the end it includes unused class names in production anyway.