3dcadmin@lemmy.relayeasy.com to Selfhosted@lemmy.worldEnglish · 3 days agoCloudflare blocking AI crawlerslemmy.sumuun.netimagemessage-square33linkfedilinkarrow-up1351arrow-down12file-text
arrow-up1349arrow-down1imageCloudflare blocking AI crawlerslemmy.sumuun.net3dcadmin@lemmy.relayeasy.com to Selfhosted@lemmy.worldEnglish · 3 days agomessage-square33linkfedilinkfile-text
Cloudflare trying to stop AI crawling somehow! https://arstechnica.com/tech-policy/2025/07/pay-up-or-stop-scraping-cloudflare-program-charges-bots-for-each-crawl/
minus-squareProntomomo@lemmy.worldcakelinkfedilinkEnglisharrow-up5·3 days agoI don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling
I don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling