Results 1 to 5 of 5
  1. #1
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    1,658
    Inktomi is crawling every page of my site, including a request for robots.txt file.

    Most of my pages have never even been submitted to them.

    Has this happened to anyone else?

    Any ideas on what is up with this?

  2. #2
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    Slurp - the Ink. spider- works just like all the other major spiders and follows links from other sites. Generally speaking you should see pages show up on MSN and, uh, whoever else they've got left within 4-6 weeks.

    Don't expect to see all of your site appear - be prepared to spend ages trying to figure out why that page four levels down with only incoming link appears and your main product page with 50 links from other pages doesn't.

    And, if you do figure this out, let me know, will ya ?

  3. #3
    Newbie
    Join Date
    January 17th, 2005
    Posts
    1,537
    Mark,

    If you could buy Inktomi, could you make a profitable search engine out of it knowing all that you do about search engines?

  4. #4
    Guest
    Deelz,
    Ink has spidered my site alot, and yes you will show up in MSN, AOL, and there are a few unfamiliar names too.

  5. #5
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    Fred,

    Could I make Inktomi profitable ? Uh, no...probably not. It seems strange that they are losing market share when you would have thought they would be best placed to increase it. After all, they are not a search engine, so are not direct competition for partners, they have the longest running and most successful paid inclusion program and so can offer revenue sharing and they have several extremely well defined and comprehensive databases of sites. Plus they've managed to cut out much of the spam that threatened to destroy them at one time.

    If I was Yahoo, for instance, I wouldn't have looked twice at Fast. If I was AOL, I wouldn't have dropped them for Google. So, would I be able to make them profitable .........no.

  6. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Deep Crawl vs What ever is going on now
    By Doug247 in forum Midnight Cafe'
    Replies: 6
    Last Post: April 16th, 2004, 12:28 PM
  2. Deep crawl
    By bpal in forum Search Engine Optimization
    Replies: 5
    Last Post: March 6th, 2004, 07:59 PM
  3. When comes the deep crawl?
    By Remon in forum Search Engine Optimization
    Replies: 13
    Last Post: January 10th, 2004, 10:41 AM
  4. Anybody else getting a deep crawl?
    By appbizz in forum Search Engine Optimization
    Replies: 3
    Last Post: January 27th, 2003, 05:12 PM
  5. Deep Crawl by Googlebot?
    By hershey in forum Search Engine Optimization
    Replies: 8
    Last Post: January 25th, 2002, 05:02 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •