Results 1 to 8 of 8
  1. #1
    Join Date
    January 18th, 2005
    Masked Referrers (no referring-URL in log)
    I just took a 100-visitor "slice of data" from our log files from this past Sunday, and made an interesting discovery: 80% of those site visitors did not show any referring URL.

    After removing search-bots and my own IP address, my log-file-slice contained requests from 97 visitors (IP addresses), and 78 of these contained no referring-URL information. As a result, I only know where 20% of my traffic is coming from.

    This is pretty bizarre -- it seems to be a change from just a month ago, but I'll need to go back and generate some very specific log-analysis reports to see what's actually going on. I'll also need to bring in my AdWords data and my SaS affiliate data to see where this might be coming from. (The strange thing is, I do have valid referrer data for every single customer who has actually placed an order -- not just this Sunday or this month, but every order we've EVER received.)

    While I'd seen a trend over the past 10 years of more and more referrer data being masked or stripped, I've never before seen this result. Are others seeing the same thing in their data?

  2. #2
    Internet Cowboy
    Join Date
    January 18th, 2005
    Bots create affiliate clicks. I have tried to ban them but that seems impossible. I have seen it over and over on several networks. From the network interface it looks like clicks on my affiliate links, shows no referring site and never generates a sale.

    There were some malicious bots that exploited CJ links a year or so ago and lowered the EPC of several merchants to 1/3 or less of their real EPC. Some of my merchants were hit by this.

  3. #3
    general fuq mrbshouse's Avatar
    Join Date
    January 17th, 2005

    have you tried to block the java bots via HTAcess?

  4. #4
    Moderator MichaelColey's Avatar
    Join Date
    January 18th, 2005
    Mansfield, TX
    There can be many different reasons for having no referring URL.

    1) Bots seldom use referring URLs. This is usually the biggest source. Sometimes they don't identify themselves in the User Agent, but their activity is usually pretty obvious. You'll usually see them hit a ton of pages and never have a referring URL.

    2) JavaScript links usually don't include a referring URL.

    3) Direct type-ins and returning from a bookmark won't include a referring URL.

    4) Links from emails usually won't include a referring URL (except from online interfaces, like

    5) Links from software applications won't include a referring URL.

    6) Some browsers let you disable (or change) the referring URL. This is not used very often, but does happen.

  5. #5
    What's the word? Rhia7's Avatar
    Join Date
    January 13th, 2006
    This problem doesn't arise from a webmaster's attempt at cloaking does it?
    ~Rhia7 -- Remember the 7
    Twitter me

  6. #6
    ABW Ambassador
    Join Date
    January 18th, 2005
    Los Angeles
    MSN Search switched over to Live and they're not sending referrers, so that could be it.

  7. #7
    I like traffic lights
    Join Date
    January 18th, 2005
    Southern hemisphere - away from Fukushima
    IIRC if you middle click in Safari to open the link in a new tab, the referrer isn't sent to the server in the request header.

  8. #8
    Newbie's Avatar
    Join Date
    May 26th, 2006
    Blocking misbehaving / bad robots to click on CPC or affiliate URLs completely is impossible, it is like spam.

    New spam of different kinds are created everyday. The Anti-spam create signatures to block them. It is a never ending process.

    New / strange robots are crawling the internet everyday... catch some, block a few, tomorrow it will be another repeat.

    MichaelColey knows his stuff. Re-read his post above.

  9. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Referring URL of a Sale?
    By spacedog in forum Commission Junction - CJ
    Replies: 12
    Last Post: November 3rd, 2006, 07:53 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts