Results 1 to 6 of 6
  1. #1
    Crazy like a fox suzigeek's Avatar
    Join Date
    January 18th, 2005
    Doh!! Blocked all Robots.
    I have a site that was doing well traffic was picking up and then all of a sudden nothing! At All!! No robots nothing but a few link requests.

    Well after thinking about it last night I decided to check my robots.txt file and sure enough... I disallowed everybody...although become bot doesn't seem to obey.

    I don't remember modifying the file. Although I do disallow everything on one of my development sites...Maybe I accidently hit the wrong button one day..

    Man 2 months or so of missed sales

    Live and learn!!

  2. #2
    ABW Founder Haiko de Poel, Jr.'s Avatar
    Join Date
    January 18th, 2005
    New York

    Does show the value of the robots.txt file tho
    Continued Success,

    The secret of success is constancy of purpose ~ Disraeli

  3. #3
    ABW Ambassador DesignerWiz's Avatar
    Join Date
    January 18th, 2005
    Ray Thomas
    Webmaster Resources:
    ABW Board Category: Programming / Coding

  4. #4
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Ooh, that's painful!
    There is no knowledge that is not power. ~Hemingway

  5. #5
    Join Date
    February 10th, 2005
    suzigeek, quick question - did you still see them in your referrer logs at all? Like would an attempt to get in show a hit from googlebot? I'm just asking because my site was down for half a day two weeks ago and I haven't had any google traffic since, but googlebot does come in every once in a while - just trying to make sure my HOST didn't somehow block google.

  6. #6
    Crazy like a fox suzigeek's Avatar
    Join Date
    January 18th, 2005
    No I saw nothing at all. Google never showed in my logs. It was a vacuum it was so dead. An occasional stragler visitor or a refferal from yahoo from a reeeaaalllly old page.

    Thats what got me wondering what happened it was unaturally not visited....and finally, finally check my robots file because thats the only thing that would keep a robot out. The site never had any funny business to get it banned even if it were banned from one SE it still would have somebody coming by. As I said the site was gaining steam and is almost a year old.

    Man I was actually happy to figure it out. At least its something within my CONTROL!!!! Unlike everything else on the web.

    Check your headers to see what they're putting out.

    I found out that my host was sending out 302redirects for awhile while it was updating something and that screwed up my traffic for a few weeks.

  7. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. No change to Robots.txt but google says pages blocked
    By JI7009 in forum Search Engine Optimization
    Replies: 7
    Last Post: November 7th, 2009, 10:34 AM
  2. DOH, CJ just went down
    By cjk in forum Commission Junction - CJ
    Replies: 18
    Last Post: October 27th, 2003, 04:11 AM
  3. DOH!
    By iucpxleps in forum Virtual Family and Off-Topic
    Replies: 16
    Last Post: August 1st, 2002, 11:51 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts