i have strong dislike how AI crawling is making legitimate crawling harder these days - the web is so great, and the metadata standards too for retrieving info like title, author etc ultimately, using cloudflare may help prevent the AI crawlers, but the AI crawler's existence is hurting the discoverability of content as a result ;(