Results 1 to 10 of 10
  1. #1
    Full Member TLE's Avatar
    Join Date
    January 21st, 2005
    Location
    Southern California
    Posts
    338
    CJ DNS Services Completely OFF CJ's Network
    Monumental day as CJ is no longer providing DNS services for its domains and DNS is now mostly provided by VC.

    Recapping recent events, a few weeks ago I had started this thread in regards to CJ's primary name server (ns2.cj.com) being unavailable. Within a few days, CJ had switched primary dns to a VC server (fs1.vclk.net), but for only the cj.com domain.

    Last Sunday afternoon/eve there was an outage at CJ, affecting all domains except for cj.com.

    Throughout yesterday, CJ/VC had switched the primary dns for all CJ domains (in addition to cj.com) to the VC name server at fs2.vclk.net. Hence, CJ is no longer using any of its server for DNS services and the work has mostly been transferred to VC's DNS servers.

    nslookup -q=soa qksrv.net ns1.mediaplex.com

    Server: fs1.snv.mediaplex.com
    Address: 64.158.223.64

    qksrv.net
    primary name server = fs1.vclk.net
    responsible mail addr = systems.valueclick.com
    serial = 2006071002
    refresh = 7200 (2 hours)
    retry = 900 (15 mins)
    expire = 604800 (7 days)
    default TTL = 600 (10 mins)
    qksrv.net nameserver = ns1.mediaplex.com
    qksrv.net nameserver = ns2.mediaplex.com
    ns1.mediaplex.com internet address = 64.158.223.64
    ns2.mediaplex.com internet address = 64.70.10.79


    Mostly?

    Thanks to ripe's Akamai reminder, There's a double lookup issue for all of CJ's tracking domains.

    In summary, CJ's tracking domains are using Akamai's DNS servers for name resolution. Unlike the standard setup where visitors are directed directly to Akamai for DNS resolution, CJ's setup has visitors going first to the VC/Mediaplex servers, which in turns point visitors to the Akamai's name servers.

    With this setup, Akamai's robustness is negated by any issue/outage from the VC DNS servers. From a performance/ speed issue, especially within the context of tracking services, the extra complete trip to lookup the Akamai host is very troubling.

    Searching for www.qksrv.net A record at g.root-servers.net [192.112.36.4]: Got referral to F.GTLD-SERVERS.net. [took 39 ms]
    Searching for www.qksrv.net A record at F.GTLD-SERVERS.net. [192.35.51.30]: Got referral to ns1.mediaplex.com. [took 72 ms]
    Searching for www.qksrv.net A record at ns1.mediaplex.com. [64.158.223.64]: Got CNAME of track.cj.akadns.net. and referral to e.root-servers.net [took 86 ms]
    Searching for track.cj.akadns.net A record at e.root-servers.net [192.203.230.10]: Got referral to H.GTLD-SERVERS.net. [took 78 ms]
    Searching for track.cj.akadns.net A record at H.GTLD-SERVERS.net. [192.54.112.30]: Got referral to asia4.akadns.net. [took 116 ms]
    Searching for track.cj.akadns.net A record at asia4.akadns.net. [61.213.147.96]: Reports track.cj.akadns.net. [took 225 ms]


    Reference

    Name Servers:
    http://www.dnsreport.com
    Enter any CJ domains and look for "fs1.vclk.net" in the INFO>SOA>Primary nameserver: box.

    Akamai/Double Lookup:
    http://www.dnsstuff.com/
    DNS Lookup - A Record. Enter www.qksrv.net, or any of the other cj tracking domains (w/ the www).
    [url="http://www.wholesaler.com"]Wholesaler.com[/url]

  2. #2
    ABW Ambassador Ron Bechdolt's Avatar
    Join Date
    January 18th, 2005
    Location
    Affiliateville, USA
    Posts
    7,927
    TLE,

    Thanks for your regular updates on this.

    I'm catching a bit of what you are saying (too techie for me) but think I am missing the point. Can you speculate on why this is all happening?
    Ron Bechdolt | Affiliate Program Management Consultant
    7 Days A Week Marketing

  3. #3
    15 years and counting
    Join Date
    January 18th, 2005
    Posts
    6,121
    They have been down quite often lately. I'm using several ISPs and they had rotating DNS problems these last few days. Everything was fine with some ISPs and down with others.
    Anyway, WHY do they have to use VC for their DNS services?
    Is it an other LMI story?

    I wonder if they are telling their merchants about the lost sales.

  4. #4
    general fuq mrbshouse's Avatar
    Join Date
    January 18th, 2005
    Location
    Argieville
    Posts
    1,381
    it seems pretty in line with the whole LMI thing....drop the data off at the front door and then continue on with what started the process (requesting a publishers page / link)

    If it does not go to a VC server first, how would they ever be able to determine the most appropriate content to display....when and IF it ever gets to that.

  5. #5
    Full Member TLE's Avatar
    Join Date
    January 21st, 2005
    Location
    Southern California
    Posts
    338
    Quote Originally Posted by Zeus
    They have been down quite often lately. I'm using several ISPs and they had rotating DNS problems these last few days. Everything was fine with some ISPs and down with others.
    Hi Zeus,
    Off topic but you should look into www.opendns.com. I've been beta testing it for a few months and it provides excellent name resolution service, instead of using your ISP's dns servers. I've known David for a few years and have been using his everydns service since 00.

    Tuan
    [url="http://www.wholesaler.com"]Wholesaler.com[/url]

  6. #6
    Full Member TLE's Avatar
    Join Date
    January 21st, 2005
    Location
    Southern California
    Posts
    338
    Quote Originally Posted by mrbshouse
    If it does not go to a VC server first, how would they ever be able to determine the most appropriate content to display....when and IF it ever gets to that.
    This is name resolution, to find the place that will determine the appropriate content, and not going to that place (VC) initially.

    Tuan
    [url="http://www.wholesaler.com"]Wholesaler.com[/url]

  7. #7
    general fuq mrbshouse's Avatar
    Join Date
    January 18th, 2005
    Location
    Argieville
    Posts
    1,381
    Tuan,
    if the search string is included in this, it would be a good quanity of data.

  8. #8
    15 years and counting
    Join Date
    January 18th, 2005
    Posts
    6,121
    Thanks Tuan,

    "OpenDNS makes the Internet experience safer, faster and smarter for you and everyone using your network."

    I'm changing my routers setting.

  9. #9
    Full Member TLE's Avatar
    Join Date
    January 21st, 2005
    Location
    Southern California
    Posts
    338
    Quote Originally Posted by 7-days
    Can you speculate on why this is all happening?
    Hi Ron,

    IMHO, CJ's management and oversight. of its technical operations can be improved upon.

    CJ's strategy seems to be that of deploying best-of-breed products, in lieu of a complete understanding of its own operations. Similar to what I was referring to here, in regards to doing whatever is necessary from a marketing standpoint to keep the accounts coming in.

    Pursuing this strategy has led CJ to the current difficult issues.

    In summary, I do not believe anyone at CJ has "ownership/management" of CJ's DNS setups. Instead, there have been various "administrators" who have made modifications to CJ's DNS records, per specific requests, but no manager who monitors and ensure the deliberate and effective delivery of DNS services.

    For example, the 12/04 outage was due to CJ effectively running on one name server, for months and without CJ's awareness.

    When CJ was aware of an issue and took actions, the actions are sometime unfortunately incomplete. After my posting in regards to primary name server ns2 being down, CJ initiated a primary server change, but only for only cj.com and not the tracking domains. Consequently, Sunday's outage affected those tracking domains which were not switched over to VC's nameservers, as was done for cj.com.

    These are straightforward outage issues, unlike what is IMHO the smoking gun as to no one is managing DNS.

    Since being a CJ affiliate, i've always wondered about the "tracking is on a separate system", posting it here as a question to Todd.

    With the recent outage, and a reminder to review Akamai's dns services for CJ, I now understand how CJ can assert tracking is on a separate system. Unfortunately though, this issue communicate a lack of management and oversight in CJ technical operations, for years ongoing.

    As a crude example, suppose you operate a retail storefront and business is booming, customers are knocking down your door. So you go out and purchase a new and much larger door and a have it installed. You get the invoice for the installation as well as periodic maintenance invoices, so you assume the door is installed and you're providing a larger doorway for customers to enter your store. There's even a counter on the new door and you can see how many people are going through it. So one day, you decide to check out your new door and see that it was installed directly behind the old door, thereby providing your customers with no larger of an entry as your old door.

    It breaks my heart every time I see this:

    Searching for www.qksrv.net A record at j.root-servers.net [192.58.128.30]: Got referral to F.GTLD-SERVERS.net. [took 365 ms]
    Searching for www.qksrv.net A record at F.GTLD-SERVERS.net. [192.35.51.30]: Got referral to ns1.mediaplex.com. [took 78 ms]
    Searching for www.qksrv.net A record at ns1.mediaplex.com. [64.158.223.64]: Got CNAME of track.cj.akadns.net. and referral to g.root-servers.net [took 83 ms]
    Searching for track.cj.akadns.net A record at g.root-servers.net [192.112.36.4]: Got referral to A.GTLD-SERVERS.net. [took 51 ms]
    Searching for track.cj.akadns.net A record at A.GTLD-SERVERS.net. [192.5.6.30]: Got referral to zb.akadns.org. [took 67 ms]
    Searching for track.cj.akadns.net A record at zb.akadns.org. [206.132.100.105]: Reports track.cj.akadns.net. [took 129 ms]

    In summary, mediaplex.com (CJ) is analogous to the old door and akadns.net (Akamai) is the new door and apparently you have to go through the old door before you can use the new door.

    I'm reminded of the Steve Jobs plea to the fist MAC team to cut one additional second of the boot time, hence giving the cumulative effect of saving the world 5 years of time. CJ receives millions of dns requests per day and assuming (conservatively) 100ms additional for each request, that works out to 1.16 day of wasted time for every million dns requests.

    I remember first seeing this a couple of years ago, and filing it away as temporary configuration. Revisiting the Akamai/dns issue this week, i'm horrified that this the de facto standard.

    Tuan
    [url="http://www.wholesaler.com"]Wholesaler.com[/url]

  10. #10
    ABW Ambassador Ron Bechdolt's Avatar
    Join Date
    January 18th, 2005
    Location
    Affiliateville, USA
    Posts
    7,927
    Tuan,

    Thanks, that really does help.

    Sounds like CJ needs to hire a dedicated contractor to help them with some doorway remodeling!
    Ron Bechdolt | Affiliate Program Management Consultant
    7 Days A Week Marketing

  11. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Affiliate Network for Services and other questions
    By thomsenhk in forum Starting an Affiliate Program & Merchant Q&A
    Replies: 5
    Last Post: February 27th, 2012, 12:29 PM
  2. Network DNS Problems - Help Please
    By ~Michelle in forum Midnight Cafe'
    Replies: 6
    Last Post: September 11th, 2004, 02:21 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •