Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The assumption is that "Google" is a good player, not a malicious one. Same idea as robots.txt. It's only there to configure the "good guys" behavior not to prevent malicious actors.

So I'm assuming Cloudflare is basically asking Google to split its crawler's UA and distinguish between search and AI overview and respect something akin ot robots.txt



Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: