Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Is it bad that I'm just as insulted by the so-called "spider trap"? It's so technologically simple as to be useless against anyone who could deploy a web scraper in the first place.

I mean, it's marked by comment tags that say "spider trap" right on them! Its the worst type of disambiguation system: likely to generate false positives, unlikely to catch real violators.



Yet the off-the-shelf bots that are just let loose on the web in general will likely fall for it, as long as the "spider trap" is not off-the-shelf itself; and the ones actually targetted at just you you likely can't defeat anyway.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: