Results 1 to 7 of 7
  1. #1
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    1,658
    Or anybody else. Just thought you would probably be the one most likely to have an idea of what could have happened here.

    On the last crawl, google only crawled the homepage of one of my sites. Didn't follow links to other pages (which had already previously been indexed), at least it didn't show up in the visitor logs. Server wasn't down at any time. Now all pages from this site, except the homepage itself, have disappeared from google index.

    That's life and I'll live with it, but I'm somewhat perplexed and hoping that there could be a reasonable explanation of how this could happen. There were no changes between crawls except removal of a few dead links, which may have already been dead on previous crawls.

  2. #2
    ABW Ambassador Abigail's Avatar
    Join Date
    January 18th, 2005
    Location
    Fraser Valley, BC Canada
    Posts
    1,100
    Are you sure your host server wasn't down? One of my sites was offline for only about 10 or 15 minutes last time and I lost almost all my pages for that site out of Google until this update - thankfully they are back!

  3. #3
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    Deelz, how many visits did you get from Googlebot ? If it was just one, this may be the cause, as it often comes by, pulls down the robots.txt file and the index page then goes away again until later in the cycle.

    There are other possibilities to do with the link structure of your site, but without seeing it, I can't comment; but the above explanation seems most likely. The cure ? Check your robots.txt to make sure that there aren't errors in it, get some extra links from sites that appear with the date next to them in Google. That should solve this problem.

  4. #4
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    1,658
    I don't use a robots.txt file - I'll have to get onto that.

    As far as linking methods, mostly plain vanilla hyperlinks to my internal pages, plus a smattering of affiliate links (CJ, Clickxchange, etc.)

    The only difference in this page since previous crawl is removal of one dead affiliate link, removal of one link to an internal page that I had eliminated, and - maybe this is what got me? - I had inserted affiliate banner code at the top of my page - BEFORE the BODY tag :rolleyes:

    Thanks

  5. #5
    Action Jackson - King of the World
    Join Date
    January 18th, 2005
    Posts
    2,201
    Excuse my ignorance, but what is that robots text file thingie?

  6. #6
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    Canada
    Posts
    1,650

  7. #7
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>I had inserted affiliate banner code at the top of my page - BEFORE the BODY tag <HR></BLOCKQUOTE>

    Ah! That may be it.

    Jackson, a robots.txt file is simply a file that allows or disallows search engine and other bots from indexing all or part of your site. Very useful for keeping googlebot away from secure parts of your site, keeping out openfind and all the other rubbish and ensuring that some spiders (like Fast) actually index your site.

  8. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. MarkyMark Chat?
    By Trust in forum Search Engine Optimization
    Replies: 7
    Last Post: January 26th, 2003, 10:42 PM
  2. Try This-MarkyMark?
    By johnniebry in forum Search Engine Optimization
    Replies: 2
    Last Post: October 1st, 2002, 05:22 AM
  3. Another question for MarkyMark... ;)
    By Taurus in forum Search Engine Optimization
    Replies: 4
    Last Post: July 7th, 2002, 06:02 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •