Results 1 to 6 of 6
  1. #1
    ABW Ambassador Ron Bechdolt's Avatar
    Join Date
    January 18th, 2005
    Location
    Affiliateville, USA
    Posts
    7,927
    How do I know that a search engine or it's spider has looked at my site? What do I look for. I use Web-stats traffic analysis program, but I can't tell which ones are the search engines. For all I know, it may not even list those. How do you all find out if they have visited?

  2. #2
    Full Member
    Join Date
    January 18th, 2005
    Posts
    462
    Hi 7-Days,

    I am not familiar with Web-Stats but there are several ways to tell if a spider has stopped by...
    Does Web-Stats allow you to view which "Browser Type" your visitors are using? If so you can go there and you will see things like: GoogleBot and Slurp which are names of bots.
    Do you have access to the access logs on your server? A lot of web hosts allow you to have access to these logs and you can find them when you FTP in to transfer files to the server. They are usually located in a folder named logs and are named by date. If you have access to these select the file in your FTP client and select "view." The file should open in notepad or wordpad. Then you will be able to scroll through it and look for the words GoogleBot or Slurp (there are other bots). You can also use the "find" command in notepad/wordpad to find the word GoogleBot or Slurp.

    Hope this helps,

    Core [img]/infopop/emoticons/icon_smile.gif[/img]

  3. #3
    ABW Ambassador Ron Bechdolt's Avatar
    Join Date
    January 18th, 2005
    Location
    Affiliateville, USA
    Posts
    7,927
    Thanks for that most helpful information. I checked my log and found several bots that had visit my site. Now if anyone can tell me which bot is from which search engine, I would really appreciate it. Here is what I found:

    Slurp
    Scrubby
    Snoopy
    Net Probe
    Scooter-3
    Sqworm
    Tooter 1
    Mercator-2
    Gigabot
    Bumblebee
    SlySearch
    Openfind
    Javal

    Thanks!

  4. #4
    Guest
    This is what I have:
    web page

  5. #5
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    Off the top of my head,

    Slurp - Inktomi
    Scrubby - Scrub the Web.com
    Scooter - Altavista's regular crawler
    Mercator - Altavista's deep crawler
    Gigabot - Gigablast.com
    OpenFind - Openfind (Chinese or Korean SE)
    Tooter - Selfpromotion.com spider

    Not sure about some of the others.

  6. #6
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    1,658
    Bumblebee is from relevare.com

    Don't know what they're all about but it looks like they've got something cooking.

  7. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Site Search Engine
    By Leeanne in forum Midnight Cafe'
    Replies: 12
    Last Post: October 29th, 2005, 05:30 PM
  2. Need a site Search Engine, Any Suggstions
    By dak142 in forum Midnight Cafe'
    Replies: 17
    Last Post: November 10th, 2004, 01:28 AM
  3. Help finding search engine for a site
    By photonstudios in forum Midnight Cafe'
    Replies: 4
    Last Post: April 23rd, 2004, 12:10 PM
  4. In Search Of A Site Search Engine Spider
    By Heidi in forum Midnight Cafe'
    Replies: 2
    Last Post: September 19th, 2002, 07:13 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •