Results 1 to 19 of 19
  1. #1
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    Haiko frequently advocates encrypting pages to prevent theft.

    I downloaded the program mentioned inThis thread
    and have been experimenting with it.

    My question is, will the encryption affect the ability of search engines to read and spider the pages?

    Thanks.

  2. #2
    Newbie
    Join Date
    January 18th, 2005
    Posts
    34
    Yes. Hell yes.

    Go to the HTMLcrypt examples page and view the source of one of the encrypted pages. Everything is one SCRIPT element. There's no human-readable text at all. Most don't even a TITLE element.

    (And it triples the byte-size of each page. Jeez.)

    Don't bother with this crap. HTML-encryption programs are written for and by half-wits. Anything you encrypt in JavaScript, I could grab a copy in 30 seconds, using only a mildly tricked-out copy of Internet Explorer.

  3. #3
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    That's what I thought. So what is a better way to keep someone from downloading my pages en-masse and uploading them somewhere else as their own? .htaccess ?

  4. #4
    Full Member
    Join Date
    January 18th, 2005
    Posts
    401
    Only thing I can think of is a mixture of flash files and html, but have first paragraph text as html for spiders?

    .htaccess wouldn't do much good I think, if the text is visible on the page it can be saved, unless in flash?

  5. #5
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    I'm not concerned about someone manually saving a few pages to their hard drive. What I want to prevent is someone downloading an entire 1000-page website, changing the PID's in the links, then uploading it as their own.

    You can use mod_rewrite in .htaccess to keep out certain browser-agents ... such as questionable spiders. I just don't know if that's the best deterrent or if there is a another way.

  6. #6
    ABW Ambassador affiliatemakeover's Avatar
    Join Date
    January 18th, 2005
    Location
    Cleveland, OH
    Posts
    821
    Unfortunately, I don't know how to stop someone from doing it either.

    But I do know of tools that do it easily, and for free. I use it with several of my clients to easily download their webtrends reports. It's actually quite convenient.

    I won't post the software link here. If you want it, pm me.

  7. #7
    ABW Founder Haiko de Poel, Jr.'s Avatar
    Join Date
    January 18th, 2005
    Location
    New York
    Posts
    21,609
    eaglefire,

    See www.prolinkz.com

    Haiko

  8. #8
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    Thanks Haiko. As far as I can tell, that script places "?" in the URLs. I'm looking for a way that won't inhibit search engines. Don't some still have issues with "?"

    I was beginning to think along the lines of using

    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>

    RewriteEngine on
    RewriteCond %{HTTP_USER_AGENT} Albert [OR]
    RewriteCond %{HTTP_USER_AGENT} Charlotte
    RewriteRule ^.*$ - [F]
    <HR></BLOCKQUOTE>

    from Islander's post in the other thread to keep out website download apps such as WebReaper and Bloodhound; while permitting access to search engine spiders. There is a list of Robots with their function here : http://www.robotstxt.org/wc/active/html/type.html

    That is, unless there is another way that will let a search engine read and spider a site while preventing theft.

    [ 06-13-2002: Message edited by: eaglefire ]

  9. #9
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    Here is a way

    Build your pages in html as you normally do but use a script called executer.cgi to call the actual merchant links.

    All of your content will still be copyable except for the links themselves. No one is going to take the time to steal all of your pages and then have to go through and manually add thousands of links having to match them all up etc.

    You can get a free copy of executer.cgi at www.smartcgis.com/executer

    What this program is or does is allows you to include pages just like shtml does but you don't need to have special .shtml extensions, you can use this with cgi or html or php whatever.

    My method of doing this is to make a page that contains your link code and call it from the executer.cgi script

    Go to www.avemall.com/abw-test.html and look at my affiliate link on that page to see this in action. Read the source code on that page to see how the link is called just like .shtml

    Search engines can not read this info so you want to only include the link with it and leave the content on your pages in html. There are a lot of other cool things this will do like include .cgi scripts on an html page or almost anything.

  10. #10
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    Thanks Heyder. I'll download it and give it a try.

  11. #11
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    Ok Heyder, I've been playing around with Executer.cgi. I can think of loads of things to use it for. Thanks for the suggestion.

    Now, as far as using it as part of a deterrent system here is what concerns me - Can't the theif just download the file(s) containing the link code along with the rest of the website, change that to his own PID, then install Executer and run the file the same way thru his own site? The name of the script as well as the file it is calling is plainly visible in the source code.

    [ 06-15-2002: Message edited by: eaglefire ]

  12. #12
    Guest
    Morning all, this may sound like a stupid question, but I am just now getting involved with this stuff.

    I use WS FTP and cannot find where to change the CModd 777/755 stuff. I found the ftp commands, but thatchoice is not in there.

    Maybe I have a limited edition?
    Is there a free ftp with these capabilities you know of I can download, or is this something thru my host?
    thanks

  13. #13
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    no it's there and should still work so long as you are on a *nix host (Unix or Linux). Windows does not support Chmod. Windows hosts that support CGI automatically set permissions to execute. If you need write access on Windows you may have to contact your sysadmin to set permissions on those files/directories.


    If you are on *nix:
    In the Remote (right-hand) window, Right click on the file or directory you want to CHMOD and just select CHMOD from the pop-up menu. A dialog will pop up asking what you want to set your permissions to.

    [ 06-15-2002: Message edited by: eaglefire ]

  14. #14
    Full Member
    Join Date
    January 18th, 2005
    Posts
    401
    Oh yeah a link redirector, good idea... did anyone ever decide if cj minded using link redirectors?

    They'd still have to put a lot of effort in for 1000 pages?... they'd be better off making their own site?... perhaps that's all you can do, make life hard for the stealer rather than a total solution?

    Also, if the cgi file's name is changed, they won't know what script you are using, that any help?

  15. #15
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>They'd still have to put a lot of effort in for 1000 pages?... they'd be better off making their own site?... perhaps that's all you can do, make life hard for the stealer rather than a total solution?

    <HR></BLOCKQUOTE>

    It could actually make it easier.
    All I would have to do is :

    a)download the link code files along with the rest of the website
    b)run execute.cgi on my own host

    It's no less trouble to change the affiliate links on 1000-plus pages, but apparently people are doing it - aren't they? There may be some limit to how large a site someone will steal b/c of the work involved, but it's actually quite easy; just use search and replace in any decent text editor.

    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>
    Also, if the cgi file's name is changed, they won't know what script you are using, that any help?<HR></BLOCKQUOTE>

    That might help but I'm not convinced it's foolproof either. There are spiders that can sniff out cgi's. If they can find and download the cgi script from the server, it won't matter what it's name is. Maybe they will use LWP:: to grab the source code from the cgi.

    Experience has taught me that if I can conceive of even a remotely possible way around something, someone else is already doing it.

    [ 06-15-2002: Message edited by: eaglefire ]

  16. #16
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    Ahh yes that is possible but I believe it as is would prevent 80% of the thieves from even bothering with it as you can't just copy and paste.

    Another thing you could do is try the includer.cgi script. It works with a link encryption tool.

  17. #17
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    I've downloaded Includer also but haven't run it yet so I wasn't aware it works with encryption. I'll check it out.

    The whole issue is rather like using a lock on your bicycle; if someone wants your bike they are going to get it, but if I can deter 80% that would make me happy. It seems the problem is an epidemic - if you have a site with any kind of ranking, it *will* be copied. Someone here talked about 500-page site being stolen and I would have thought that to be more trouble than a theif would want, so I have to assume size doesn't matter.

    Probably a combination of using mod_rewrite to deny access to known website downloaders like RoboFox and WebReaper, along with link re-direction techniques would be the most effective approach.

  18. #18
    Newbie
    Join Date
    January 18th, 2005
    Posts
    2,694
    you know, it might be a good idea if we started gathering some data on Xeroxing incidents - size of site, file extensions most often copied, log activity - and maybe come up with some guidelines for theft-proofing.

  19. #19
    ABW Ambassador webmarm's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,713
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Someone here talked about 500-page site being stolen and I would have thought that to be more trouble than a theif would want, so I have to assume size doesn't matter.
    <HR></BLOCKQUOTE>

    Okay, now you got my attention :eek: . I had also thought that my larger sites would be too much trouble for thieves. I guess I also assumed that most site thieves were just this side of stoooopid and would go for the smaller not so complexly linked sites. Now I am reminded about that saying about the word "assume".

    1) I messed up my mod_rewrite once and almost got kicked off my server. .htaccess files need to be written exactly correctly.

    2) There is always the next line of site download clients.

    3) I agree with your bike lock post a lot, eaglefire. Let's get a Krypton lock
    [img]/infopop/emoticons/icon_wink.gif[/img] .

  20. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Removing pages from search engines?
    By Verbalkent in forum Newbie Affiliate FAQs & Helpful Articles
    Replies: 7
    Last Post: August 16th, 2004, 04:43 PM
  2. Naming pages for search engines
    By Vinny O'Hare in forum Search Engine Optimization
    Replies: 10
    Last Post: September 8th, 2003, 02:08 AM
  3. Dynamic Pages and Search Engines
    By Haiko de Poel, Jr. in forum Midnight Cafe'
    Replies: 1
    Last Post: August 5th, 2002, 11:59 PM
  4. Dynamic Pages and Search Engines
    By Haiko de Poel, Jr. in forum Search Engine Optimization
    Replies: 3
    Last Post: August 5th, 2002, 04:28 PM
  5. Dynamic Pages and Search Engines
    By Haiko de Poel, Jr. in forum Programming / Datafeeds / Tools
    Replies: 0
    Last Post: August 5th, 2002, 12:53 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •