Blocking spam bots from accessing your site

In Archive by Fredy Ore

Mark Pilgrim comments on a technical post on how to block spam automated robots and spiders from accessing your site & content and eating your bandwidth. He comments on how some grab whatever they want leave and do so without respecting the Robot Exclusion Standard which has been in affect for almost 10 years. :::