Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

No, but in whatever little bot/scraper blocking work I did, user-agent blocking was an incredibly powerful low-hanging fruit.

For sensitive material, we would often insta-ban anyone who hit us with a 'curl' user-agent and that definitely killed a lot of script-kiddies' dreams right then and there.

Obviously, if you are a determined "hacker" you would get past that and hit the honeypots but simple filters could filter out enough scrapers such that the rest of the attacks could be manually evaluated by humans.



It feels very strange to me that someone can honestly believe that google isn't pulling shenanigans.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: