Results 1 to 5 of 5
  1. #1
    The Great Egress NewcastleB's Avatar
    Join Date
    January 19th, 2005
    Posts
    65
    Slurp Making Strange Requests
    One of my sites is getting some weird requests from the slurp bot. It's requesting pages like:
    /SlurpConfirm404/atmosphere_f.htm
    /SlurpConfirm404/duilio/allaboutme.htm
    /SlurpConfirm404/Honeymoon.htm

    These pages don't exist and have never existed. And I've certainly never had a directory called SlurpConfirm404. The ip address points to Inktomi Corporation, so it's not a rouge bot masquerading. Anybody else seeing this or have any ideas?
    But are you still master of your domain?

  2. #2
    Lite On The Do, Heavy On The Nuts Donuts's Avatar
    Join Date
    January 18th, 2005
    Location
    Winter Park, FL
    Posts
    6,930
    googled the first page name and looked at the cache of the result there...
    http://www.google.com/search?hl=en&q=atmosphere_f.htm

    maybe someone there (and elsewhere) scraped / linked to your site or something and now slurp bot is visiting to confirm it's a 404 error page still or not or whatever...

  3. #3
    The Great Egress NewcastleB's Avatar
    Join Date
    January 19th, 2005
    Posts
    65
    Thanks Donuts, I just did the same thing, and came back to edit my post. That'll teach me to post first and search second.

    D'oh I really feel like and idiot now! I just glanced at Donuts' post and thought we searched for the same thing. I searched for "SlurpConfirm404/" and found my way to a page on Yahoo with the explanation. If anybody else is seeing this, here's what they say:

    Some web servers send a site navigation page or other response page with a "HTTP 200 OK" response instead of a "HTTP 404 Not Found" result for page-not-found conditions. To check on web server handling of page-not-found conditions, Slurp will occasionally send deliberately odd URLs built from random words to sites from which no 404 results have been seen. These URLs are built intentionally to not match any actual content at the site. We save information on the web server response to requests for non-existent pages so we can correctly recognize and remove obsolete URLs in our search database.

    A Slurp check for 404 results from a web server consists of requests for up to 10 such URLs. The check for 404 behavior is not a normal part of Slurp site refresh, so such requests will be rare.
    Last edited by NewcastleB; June 28th, 2006 at 02:25 PM. Reason: Me Posting without reading the whole post
    But are you still master of your domain?

  4. #4
    Lite On The Do, Heavy On The Nuts Donuts's Avatar
    Join Date
    January 18th, 2005
    Location
    Winter Park, FL
    Posts
    6,930
    Ah, cool, so they check to see how you would handle a 404 - then compare that to other responses, as needed - so they can know that your typical 404's, looking like what they know are real 404, become more meaningful.

    Anticipating poor webmaster skills - good.

    Obviously, you're not in that category. Not only do you likely have proper 404 responses NewcastleB, but you comb through your logs and figure out things you don't understand. Kudos for you, webMASTER.

  5. #5
    notary sojac Herb ԿԬ's Avatar
    Join Date
    January 18th, 2005
    Location
    Central/Western NY State
    Posts
    7,741
    Cool
    cool

  6. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Yahoo Slurp
    By Trust in forum Search Engine Optimization
    Replies: 10
    Last Post: February 17th, 2004, 01:27 PM
  2. Slurp?
    By frank3iii in forum Search Engine Optimization
    Replies: 1
    Last Post: September 6th, 2003, 10:20 AM
  3. YahooSeeker / Slurp
    By ellen-s4y in forum Search Engine Optimization
    Replies: 1
    Last Post: September 5th, 2003, 10:12 AM
  4. Inktomi Slurp...
    By Electropulse in forum Search Engine Optimization
    Replies: 5
    Last Post: July 31st, 2003, 02:44 PM
  5. Where did slurp go?
    By AffiliateBuddha in forum Search Engine Optimization
    Replies: 2
    Last Post: April 17th, 2003, 02:29 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •