Results 1 to 5 of 5
  1. #1
    Member
    Join Date
    January 18th, 2005
    Posts
    154
    Today I noticed a user agent by the name of LinkWalker when I checked my server logs. It seems to be very voracious. It visited 162 pages before calling it quits. My question is, what is LinkWalker?

    Is it a soft, cuddly, harmless spider that needs to be embraced as it means no harm?

    Or (more likely) is it a tool used by site theives, spammers and the like? A truly evil spider that needs to be banned to the dark recesses of the internet until the end of time.

    Greg

  2. #2
    Member
    Join Date
    January 18th, 2005
    Posts
    154
    Wow

    The above post is only #38 for me. Not real good considering I've been an ABW member for about two years.

    I guess I'm part of the silent majority

    Greg

  3. #3
    Affiliate Manager Allen Nance's Avatar
    Join Date
    January 18th, 2005
    Location
    Colorado River, Bullhead City AZ
    Posts
    1,604
    Luke's Son

  4. #4
    Member
    Join Date
    January 18th, 2005
    Posts
    154
    While on the subject, does anybody know of a list of bots that could be added to robots.txt? Too many hits from these things sure makes server logs hard to read.

    I've even been tempted to try blocking the CJ spider, but am afraid what the potential consequences could be.

    Greg

  5. #5
    You are in, or you are out ... choose!
    Join Date
    January 18th, 2005
    Posts
    459
    http://www.robotstxt.org/

    dWoz - serious webmaster resources.

  6. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Linkwalker??
    By Catwoman in forum Midnight Cafe'
    Replies: 3
    Last Post: August 4th, 2004, 07:14 PM
  2. LinkWalker?
    By netsu in forum Search Engine Optimization
    Replies: 3
    Last Post: May 11th, 2004, 03:26 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •