Results 1 to 1 of 1
December 29th, 2007, 12:46 AM #1
Fun & Games this month - spiderbots
- Join Date
- January 17th, 2005
I've been having nothing but fun this month with 3 spiderbots coming after some of my domains - SBIderbot, Twiceler, and MJ12bot have been attacking some of my domains at the rate of one or more per second.
My robots.txt files specifically tell them to go away - they read but ignore this file.
I tried to ban their specific IP# - they changed IP#
Googling a bit I found other souls being attacked and finally found a solution that works for me (so far...)
I enter the Apache configuration file and instead of banning a specific IP or range of IP's, I ban the signature name for the bots.
SetEnvIfNoCase User-Agent "^MJ12bot(.*)1\.0\.8" stay_out0
Deny from env=stay_out0
Allow from all
They give me a bunch of ERROR 403 Forbidden's but I can ignore these
By gold in forum Programming / Datafeeds / ToolsReplies: 1Last Post: October 15th, 2001, 09:37 PM