Results 1 to 2 of 2
  1. #1
    Sgt. Joe Friday frank3iii's Avatar
    Join Date
    January 18th, 2005
    Posts
    441
    Does anyone know the difference in slurp/si and slurp/cat?

    Seems like they come to the site near the same time, read the robots.txt, read the index then come back in about a week.

    Frank

    "Just the facts, Ma'am"
    Sgt. Joe Friday, Dragnet

  2. #2
    Full Member ellen-s4y's Avatar
    Join Date
    January 18th, 2005
    Posts
    489
    I think slurp/si is for robots.txt and slurp/cat is for everything else.

    I noticed yahooseeker get robots.txt with 66.196.93.5 and crawls everything else with 66.196.93.30

  3. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Silly Yahoo! Slurp
    By Snib in forum Search Engine Optimization
    Replies: 0
    Last Post: August 12th, 2004, 04:25 PM
  2. Yahoo Slurp
    By Trust in forum Search Engine Optimization
    Replies: 10
    Last Post: February 17th, 2004, 01:27 PM
  3. YahooSeeker / Slurp
    By ellen-s4y in forum Search Engine Optimization
    Replies: 1
    Last Post: September 5th, 2003, 10:12 AM
  4. Inktomi Slurp...
    By Electropulse in forum Search Engine Optimization
    Replies: 5
    Last Post: July 31st, 2003, 02:44 PM
  5. Where did slurp go?
    By AffiliateBuddha in forum Search Engine Optimization
    Replies: 2
    Last Post: April 17th, 2003, 02:29 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •