Results 1 to 2 of 2
  1. #1
    Full Member
    Join Date
    January 17th, 2005
    Seems Inktomi decided that my pages were worth looking at.. it now goes further than just hitting robots.txt and has checked out about 4 pages.

    that's over 2 days :P fairly slow spider.

    I read that when you pay for inclusion for a page on your site.. then tell the spider to fetch a different page then your first designated page is dropped from the index and the spider concentrates on the new one.

    If you do not renew pay for inclusion and your page(s) get dropped.. has there been any problems getting inktomi to come along and index any previously paid pages for free?

  2. #2
    Affiliate Miester my2cents's Avatar
    Join Date
    January 18th, 2005
    far far away....
    I dont pay for inclusion for any reasons...

    take Inktomi for example.. I have one site that has 5000 hits from the Inktomi spider that is Jan only... this spider has been hiting this site for 6 months...

    I am also listed in Yahoo... like wise I did not pay for inclusion I have gotten 100 hits for Yahoo this month...

    I dont know what all the fuss is about...

    I personally like the second and third tier SE
    they provide me with more traffic then all the other SE combined except Google...

    I havent done anything special except write pages that have good content...


    And, that's the bottom line because it's my2cents!

  3. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Blocking bots from spidering
    By beggers in forum Scripts
    Replies: 7
    Last Post: September 19th, 2006, 11:17 AM
  2. Time between spidering and indexing
    By hellojello134 in forum Midnight Cafe'
    Replies: 1
    Last Post: July 13th, 2005, 05:48 PM
  3. Slow down spidering
    By dollardaze in forum Scripts
    Replies: 2
    Last Post: November 18th, 2004, 06:07 AM
  4. Google Spidering...
    By eggerda in forum Search Engine Optimization
    Replies: 14
    Last Post: September 15th, 2003, 12:44 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts