Results 1 to 3 of 3
  1. #1
    Internet Cowboy
    Join Date
    January 18th, 2005
    Posts
    4,662
    If I use robots.txt to exclude a certain script from being indexed, will a URL containing that script then be excluded?

    Example: I exclude rrr.php

    Would the URL:

    xxxxx.com/rrr.php?id=sssss&item=DDDDD......

    be excluded? I want the bot to follow the link, but not the script itself.

    Any ideas?

    Thanks


  2. #2
    ABW Ambassador Nature Boy's Avatar
    Join Date
    January 18th, 2005
    Location
    Tennessee
    Posts
    1,423
    I'm not sure myself, but just to be on the safe side, I always put my redirect scripts in a directory by itself and use robots.txt to keep them out of it. This keeps the bots from following the link.

    But of course, that's not what you asked
    Scott
    If you can't dazzle them with brilliance, then baffle them with bulls#!t
    Don't tell me that you'll do it... SHOW ME.
    Just because everyone else is drinking it is no reason for me to drink the KOOL-AID.

  3. #3
    Full Member
    Join Date
    January 18th, 2005
    Location
    Tamborine Mountain Australia
    Posts
    248
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by scohaz:
    If I use robots.txt to exclude a certain script from being indexed, will a URL containing that script then be excluded?

    Example: I exclude rrr.php

    Would the URL:

    xxxxx.com/rrr.php?id=sssss&item=DDDDD......

    be excluded? I want the bot to follow the link, but not the script itself.

    Any ideas?

    Thanks <HR></BLOCKQUOTE>
    That's an interesting question...
    I could be completely wrong here but it seems to me that excluding rrr.php would in fact do what you want and just exclude the script itself. I do know that many people have trouble excluding dynamic URLs using a robots.txt file and if it was just a case of excluding the first part of the dynamic URL surely there would be no issue with this at all.

    It makes sense to me that it would work.
    [left][font=Arial]"If you come to a fork in the road, take it" - Yogi Berra [/font][/left]

  4. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Looking for a Good Robots.txt file
    By Doug247 in forum Midnight Cafe'
    Replies: 8
    Last Post: July 24th, 2007, 06:18 PM
  2. editing Robots.txt file
    By KODea in forum Search Engine Optimization
    Replies: 5
    Last Post: August 28th, 2006, 03:02 PM
  3. Do I need a "robots.txt" file?
    By unclejesse in forum Search Engine Optimization
    Replies: 20
    Last Post: June 12th, 2004, 11:02 AM
  4. robots.txt file question
    By Doc Sawyer in forum Search Engine Optimization
    Replies: 4
    Last Post: December 7th, 2002, 07:10 PM
  5. robots.txt file????? HELP
    By john dundas in forum Search Engine Optimization
    Replies: 4
    Last Post: August 18th, 2002, 05:36 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •