Results 1 to 9 of 9
  1. #1
    pph Expert! Gordon's Avatar
    Join Date
    January 18th, 2005
    Location
    Edmonton Canada
    Posts
    5,781
    how can I do this please?
    How can I stop these type of guys/bots from following my cloaked links and elevating my click throughs please?

    They have a lot of IP's ranging between 64.62.168.0 - 64.62.168.127 (I got this info from doing a who-is search)

    I have a cloaked URL and it goes to a separate directory (say it is called "links") I have put some of the IP's on my robots.txt to keep them out of the "links" directory but they still follow through and elevate my click throughs. This is what I have on my robots.txt
    Disallow: /links

    User-agent: 64.62.168.70
    Disallow: /

    User-agent: 64.62.168.64
    Disallow: /

    etc.

    and a few others that they seem to use most.

    My questions are is there a way to ban the whole block and what can I do if they take no notice of the robots.txt which they have not been doing for the past few months.

    Thanks in advance for any help on this.
    One day parasites and their ilk will be made illegal, I bet a few Lawyers will be pissed off when the day comes.
    Mr. Spitzer is fetching it nearer

    YouTrek

  2. #2
    ABW Ambassador Nature Boy's Avatar
    Join Date
    January 18th, 2005
    Location
    Tennessee
    Posts
    1,423
    This is what I use Gordon.

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /images/
    Disallow: /links/

    This keep all robots.txt obeying bots from following "cloaked" links. If that doesn't work, then add this to you're .htaccess file.

    order allow,deny
    allow from all
    deny from 64.62.168

    This should keep Gigablast from poking around your site. It ain't a perfect solution, but it works.
    Scott
    If you can't dazzle them with brilliance, then baffle them with bulls#!t
    Don't tell me that you'll do it... SHOW ME.
    Just because everyone else is drinking it is no reason for me to drink the KOOL-AID.

  3. #3
    pph Expert! Gordon's Avatar
    Join Date
    January 18th, 2005
    Location
    Edmonton Canada
    Posts
    5,781
    Thanks Scott. I've tried that now I'll check tomorrow and see if the IP's I blocked are showing after tonight.

    Is it possible to just ban a block of IP's like in this example this guy (its either NJ Tech Solutions or Hurricane Electric) only uses the IP's between 64.62.168.0 and 64.62.255.255 Is it possible to ban them yet still allow the IP's from 64.62.0.0 to 64.62.167.255 to access the directory?
    One day parasites and their ilk will be made illegal, I bet a few Lawyers will be pissed off when the day comes.
    Mr. Spitzer is fetching it nearer

    YouTrek

  4. #4
    pph Expert! Gordon's Avatar
    Join Date
    January 18th, 2005
    Location
    Edmonton Canada
    Posts
    5,781


    I would love some input on this please
    One day parasites and their ilk will be made illegal, I bet a few Lawyers will be pissed off when the day comes.
    Mr. Spitzer is fetching it nearer

    YouTrek

  5. #5
    ABW Ambassador Nature Boy's Avatar
    Join Date
    January 18th, 2005
    Location
    Tennessee
    Posts
    1,423
    Quote Originally Posted by Nature Boy
    ...add this to you're .htaccess file.

    order allow,deny
    allow from all
    deny from 64.62.168
    64.62.168.0 thru 64.62.168.127 is Gigablast "indexing" your site. Putting the above into your .htaccess will keep bots/anyone with an IP from 64.62.168.0 thru 64.62.168.255 from accessing your site, which included Gigablast's bots. I don't think I would go so far to ban everyone from 64.62.0.0 to 64.62.167.255.
    Scott
    If you can't dazzle them with brilliance, then baffle them with bulls#!t
    Don't tell me that you'll do it... SHOW ME.
    Just because everyone else is drinking it is no reason for me to drink the KOOL-AID.

  6. #6
    pph Expert! Gordon's Avatar
    Join Date
    January 18th, 2005
    Location
    Edmonton Canada
    Posts
    5,781
    Thanks Scott I used this and it stopped them dead in their tracks.

    I guess I will just have to make a long list of the denied IP's as I dont want to block the IP's after 64.62.168.127.
    One day parasites and their ilk will be made illegal, I bet a few Lawyers will be pissed off when the day comes.
    Mr. Spitzer is fetching it nearer

    YouTrek

  7. #7
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    Oh, it's Gigablast that's causing the irritation? Gigabot seems to obey my robots.txt. I have it set up like this, though, NOT by IPs:

    User-agent: Gigabot/2.0
    Disallow: /

    That has kept the thing out, and since they weren't sending me traffic anyway, it's no loss.
    There is no knowledge that is not power. ~Hemingway

  8. #8
    pph Expert! Gordon's Avatar
    Join Date
    January 18th, 2005
    Location
    Edmonton Canada
    Posts
    5,781
    that might be why mine is not keeping them out Leader I will add that to it thanks
    One day parasites and their ilk will be made illegal, I bet a few Lawyers will be pissed off when the day comes.
    Mr. Spitzer is fetching it nearer

    YouTrek

  9. #9
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    It will recheck the robots.txt a lot at first, (hoping it changes back, I guess) but those robots.txt checks are all you should see from that bot once you make the change. And eventually it'll realize you really, really mean it after all, and only check occasionally after that
    There is no knowledge that is not power. ~Hemingway

  10. Newsletter Signup

+ Reply to Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •