Results 1 to 4 of 4
April 1st, 2005, 09:24 AM #1Banning a bot
I want to completely ban Yahoo from crawling on of my sites. What do I need to pu in the robots.txt?
I don't want the image bot, merchant bot, or any other bot related to yahoo crawling my site.
April 1st, 2005, 09:33 AM #2
April 1st, 2005, 12:34 PM #3
Thanks. Yahoo takes about 3 GB a month but they booted me from thier index back in DEC. 04. I'm actually trying to disallow them for a while to see if I can get back in after a month or so.
April 1st, 2005, 08:24 PM #4
China, Yahoo, eBay ???
- Join Date
- January 18th, 2005
- Texas, USA
I've been having problems with some kind of bot from China pounding my site apparently on behalf of Yahoo and eBay. Close to 2,000 hits last month.
I looked up the IP# in my Latest Visitors Apache web server log several days ago and entered it into my Apache IP Deny Manager. It seemed to stop the hits.
Now, several days later, it has switched to another IP# which I promptly entered into my Apache IP Deny Manager. 120 pages and 120 hits today.
Could it be some kind of crazed programming error?
Does one have ANY idea what is with China, Yahoo and eBay? I emailed
the hostmaster in China inquiring about the matter but never recieved a reply.
By Doug247 in forum Midnight Cafe'Replies: 2Last Post: April 1st, 2005, 12:34 PM