Results 1 to 3 of 3
  1. #1
    Member
    Join Date
    January 18th, 2005
    Posts
    134
    Wow, that was quick. Here we have an update going on and the intrepid bot is scurring about at the same time. Grabbed a bunch of pages today. Could we be seeing more frequent updates?

  2. #2
    Member
    Join Date
    January 18th, 2005
    Posts
    136
    That is a common misconception. Somehow people think that Google crawls 2 or 3 days per month and then 2 or 3 weeks later updates the index.

    Instead, think about it this way. Google has three kinds of servers:

    1. The servers which serve up the web pages for us from the "current index".

    2. The servers which crawl the internet grabbing pages and pre-processing them.

    3. The servers which create the final index from the pages that were grabbed and pre-processed by server type 2.

    There are several of each type of server (they have the largest known server farm the last I heard). Server type #1 is always running although different servers may be pointing at different versions of the indexes created by server type #3. This is why www, www2 and www3 often produce different results during the so-called "Google Dance".

    Server type #2 runs continually. When it is "done" with a pass, it just starts over. The time of the month that GoogleBot (server type #2) hits your site is merely random. It doesn't mean that GoogleBot was running at that time of the month and no other time. This type of server runs continually.

    The fact that a server type #2 hit your site while the results of a server type #3 where still being distributed among the servers doesn't really mean much of anything.

  3. #3
    Member
    Join Date
    January 18th, 2005
    Posts
    134
    Great! I'll accept a crawl from Google at any time (except when my host is down).

  4. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. What's this Googlebot?
    By Leader in forum Search Engine Optimization
    Replies: 5
    Last Post: June 24th, 2003, 01:32 PM
  2. googlebot
    By cowox in forum Search Engine Optimization
    Replies: 16
    Last Post: September 4th, 2002, 09:05 AM
  3. GoogleBot .. where is he?
    By cowcool in forum Search Engine Optimization
    Replies: 19
    Last Post: July 4th, 2002, 11:33 AM
  4. Is This Googlebot
    By Abigail in forum Search Engine Optimization
    Replies: 2
    Last Post: June 26th, 2002, 10:51 AM
  5. Googlebot
    By hershey in forum Search Engine Optimization
    Replies: 18
    Last Post: January 16th, 2002, 05:41 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •