Results 1 to 7 of 7
  1. #1
    Full Member
    Join Date
    January 18th, 2005
    Posts
    236
    Hi
    On my site tracking software I noticed that I have twiced been visited by

    spider.wlf023proxy-aol.com

    The thing is that my site has not been listed on AOL. Can anyone shed any light on this.

    Also can someone, in plain english, explain extacly what SPIRDERS are. I have a rough idea but it would be nice to read some info on it and no doubt others will, even if started as a new topic.

    regards John

  2. #2
    Full Member
    Join Date
    January 18th, 2005
    Posts
    270
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>spider.wlf023proxy-aol.com<HR></BLOCKQUOTE>

    It's probably a spider of some sort, but more than likely, it's an aol user asking for some information from aol, and aol has decided to cache your page, or is updating a cached page that it has.

    I'd just ignore it unless the bandwidth burden becomes excessive.

    As for what a 'spider' really is: as a most simplistic answer, it's a computer program that is designed to index URLs by following any and all hypertext references that it finds or is told to find. It's basically an indexing program.

    The spider 'finds' URLs, passes the information on to an indexing program that adds the information into a huge file that allows people to ask questions of it. So, if you've put the right answers in your hypertext, and the people asking questions of the huge file ask the right questions, the indexing program should pass back your hypertext as an answer.

    CodeJockey.

  3. #3
    Full Member
    Join Date
    January 18th, 2005
    Posts
    347
    The spiders work their way (crawl) on the world wide web. They follow the links they find, always hunting for food (content and keywords).

    Unlike normal spiders, the web-spiders don't consume what they find, but rather try to remember what it found where, in case they need to serve it to us when we search.

    The AOL-spider probably found a link to your site on another site it was crawling.
    (If it was a spider)

    [ 01-14-2002: Message edited by: OveB ]

  4. #4
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    LOL, Jockey, he said "English", lol!

    A spider is a program that indexes what it finds on the web. It may make a record of entire web pages, or just look for certain elements and then move on. It goes from page to page via links.

    Then the info it gets goes into a database, usually at a search engine.

    Then when people type in a search term, if your site matches the criteria set by the engine's program, it should come up.

    The more the program which runs the database figures that your site matches, the higher your site should rank.

    Just how the program determines which site should rank highest, varies by search engine.

    Some spiders DON'T go with search engines. There are spiders which are programmed to take note of e-mail addresses found at sites. There are spiders which steal sites. There are spiders which steal graphics, and those that just index graphics at search engines. I'm sure there are other uses for spiders, too--some good and some bad.

    HTH

    [ 01-14-2002: Message edited by: Leader ]

  5. #5
    Guest
    spider.wlf023proxy-aol.com is not a spider. Everyone using AOL as ISP will show up as spiderxxxxxxxxxaol.com in your stats.


    Quiet777

  6. #6
    ABW Founder Haiko de Poel, Jr.'s Avatar
    Join Date
    January 18th, 2005
    Location
    New York
    Posts
    21,609
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>spider.wlf023proxy-aol.com is not a spider.<HR></BLOCKQUOTE>

    Yep

    Haiko

  7. #7
    Full Member
    Join Date
    January 18th, 2005
    Posts
    236
    Hello Folks,

    Thanks for all the replys its much appreicated. I know a lot more about spiders than I did yesterday. Also now know what to look for when aol user visits.

    thanks guys

    regards John

  8. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. AOL Search Marketplace - Buy Adwords Directly From AOL
    By Trust in forum Search Engine Optimization
    Replies: 3
    Last Post: April 9th, 2007, 06:45 PM
  2. good spider or bad spider?
    By Gordon in forum Newbie Affiliate FAQs & Helpful Articles
    Replies: 3
    Last Post: September 3rd, 2004, 09:38 AM
  3. AOL Time Warner Moves to Eliminate AOL From Name
    By Adam Ward in forum Midnight Cafe'
    Replies: 2
    Last Post: August 11th, 2003, 04:01 PM
  4. AOL spinning off from AOL Time Warner ?
    By Andy Rodriguez in forum Midnight Cafe'
    Replies: 0
    Last Post: May 28th, 2003, 12:35 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •