Results 1 to 16 of 16
  1. #1
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Accessing CJ datafeeds
    First, I'll give CJ some credit: CJ has definitely "turned around" its handling of merchant datafeeds for affiliates. I requested access to datafeeds, and 48 hours later I received a reply telling me to identify no more than 5 merchants whose datafeeds I wanted, and referring me to CJ's datafeed specifications. It then took only 4 days before the datafeeds were posted for me to download.

    Unfortunately, I now realize that I will be unable to use CJ datafeeds without incurring substantial additional expense to have someone write a complex script; CJ's datafeed delivery is quite "different."

    The complexity comes from the fact that CJ's datafeeds are combined into a single ZIP file, preventing automation using Datafeed Studio (or any other datafeed script that I'm aware of). Inside the ZIP file are a number of merchant datafeed files, each named with the merchant ID, merchant name, and then an appended date.

    I'm "bumping" CJ datafeeds (and thus all CJ merchants) back to the bottom of my project queue -- there are many other higher-priority tasks that deserve my time.

  2. #2
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    You can request that CJ rename the zip file to something like catalog.zip. Then you can write a very simple PHP or shell script that downloads it and unzips it to your server. Then you only need to configure your software to import the datafeeds locally from your server.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  3. #3
    Full Member
    Join Date
    October 22nd, 2006
    Posts
    200
    Quote Originally Posted by markwelch
    or any other datafeed script that I'm aware of
    I would be interested to know how many of the available scripts (apart from Datafeed Studio) you have actually used to make such a sweeping statement.

    Zip functions are readily available in php and it's not rocket science to move a CJ database or any other database for that matter into a standard format. Perhaps the only one I am aware of that may cause a problem is Clixgalore because they don't bother even to have a product id in the feed.

    Bob

  4. #4
    Full Member iolaire's Avatar
    Join Date
    October 3rd, 2006
    Location
    Arlington, VA
    Posts
    229
    markwelch - This will not be a direct help, but here is the ruby code to download the file. I'm sure you could convert it to asp/php fairly quick:
    Code:
           require 'net/ftp'
           ftp = Net::FTP.new('datatransfer.cj.com','123456','mYhappyPassword')
           ftp.passive = true
           ftp.chdir('outgoing')
           ftp.chdir('productcatalog')
           ftp.chdir('123456')
           ftp.getbinaryfile( file_name, "/downloaddir/" + file_name + "", 1024)
           ftp.close
    You would then unzip the file.. Once that is done I'd assume that you would process those files in datafeed studio in what ever fashion you do other feeds.

  5. #5
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Donk, It's not rocket science; as you know, however, it does require more LAMP and Datafeed Studio knowledge than I currently have.

    Datafeed Studio appears to require either that the file be accessible as a text file or as a single-file-zip-file on a remote server, or that the file be renamed and stored locally with a specific filename, where the file is then treated as a "manual" import (I'm not sure how this "manual import" process might work). I've previously only used datafeeds that were accessible as a separate file (text or zip), so this is new to me.

    I'm not a PHP/LAMP person; I'm sure that PHP includes ZIP functionality, but I can't use it without lots of help -- which will also need to include dealing with file-processing and file-naming conventions not only at CJ, but also within Datafeed Studio.

    Also, Donk: I'd love to hear of other scripts that can handle this complexity without additional programming. I certainly haven't tried many -- indeed, most script authors have told me directly that their current products cannot meet my needs. (I can't find any indication that [i]your[/] script can import CJ feeds, but perhaps it can.)

    Thus far, my understanding is that Datafeed Studio is the most advanced available script for importing datafeeds. If I'm mistaken, please point me to another tool that will currently better meet my needs.

    As I've made clear by my repeated attempts to hire a developer to implement a custom solution that I might actually understand, I don't really like DS all that much and would gladly switch if I could identify another tool that actually worked better for my needs.

    iolaire, I appreciate your posting your sample Ruby code, but I can't translate it into PHP or ASP. I'm paying a programmer to do the work for me, including paying him to try to figure out how to "work around" the many limitations of Datafeed Studio. I have a long list of higher priority tasks I need him to work on, if he agrees to continue working for me.

    CJ is not the only affiliate-technology-provider that delivers datafeeds in a way that I can't currently use. Brideluxe, for example, combines multiple merchant datafeeds into a single file (CJ can also do this), but Datafeed Studio requires that each merchant's datafeed be stored in a separate file.
    Last edited by markwelch; July 19th, 2008 at 04:57 PM.

  6. #6
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Another solution that shouldn't require any additional work is to configure a single CJ merchant as an external single-zip with the file and credentials provided by CJ. Then configure all the others as external text files with URLs pointing back to the unzip location on your server. This is what I do and it works fine. I'm sure most datafeed import scripts can handle this with no modifications. The only thing you need to do is make sure the name of the zip file is the same every time.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  7. #7
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Snib, I don't understand what you're saying. CJ's specs say that if I'm going to download feeds from their server, then they will merge them into a single zip file. Datafeed Studio offers no documentation regarding downloads of zip files or files inside zip files, although I believe that if I tell it to import a zip file, it will automatically unzip the file and try to import an identically-named file inside (which CJ doesn't seem to offer as an option). I'm not sure what you mean when you suggest that I set up Datafeed Studio to "point back to the unzip location on your server" (it apparently can only load datafeeds via "wget" from a remote server; nor do I know what the "unzip location" might be.

    Perhaps you are referring to their "push" solution which would upload feeds to my ftp server; again, since I'm not a LAMP person, I'm not comfortable trying to set up and manage an ftp server for this purpose. Perhaps that will prove to be the best solution to implement, but if so, then it's quite some time away.

    I'm sorry I bothered to raise this issue -- apparently, anyone who wants CJ datafeeds needs to have a lot more programming knowledge than I have. No matter which of CJ's options I choose, I must pay to create a custom script to deal with the CJ feed.
    Last edited by markwelch; July 19th, 2008 at 04:59 PM.

  8. #8
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Quote Originally Posted by markwelch
    Snib, I don't understand what you're saying. CJ's specs say that if I'm going to download feeds from their server, then they will merge them into a single zip file. Datafeed Studio offers no documentation regarding downloads of zip files or files inside zip files, although I believe that if I tell it to import a zip file, it will automatically unzip the file and try to import an identically-named file inside (which CJ doesn't seem to offer as an option). I'm not sure what you mean when you suggest that I set up Datafeed Studio to "point back to the unzip location on your server" (it apparently can only load datafeeds via "wget" from a remote server; nor do I know what the "unzip location" might be.

    Perhaps you are referring to their "push" solution which would upload feeds to my ftp server; again, since I'm not a LAMP person, I'm not comfortable trying to set up and manage an ftp server for this purpose. Perhaps that will prove to be the best solution to implement, but if so, then it's quite some time away.

    I'm sorry I bothered to raise this issue -- apparently, anyone who wants CJ datafeeds needs to have a lot more programming knowledge than I have. No matter which of CJ's options I choose, I must pay to create a custom script to deal with the CJ feed.
    I was assuming Datafeed Studio lets you configure the zip file name and the unzipped file name. If this is possible you can tell it that MerchantX has the zip file provided by CJ and the actual datafeed is in MerchantX-file.txt. Datafeed Studio should download the zip, unzip it and import MerchantX. As a side effect you've now got all the other datafeeds unzipped on your server as well. Then you can just configure your software to import those as if they were never in a zip because they're already on your server.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  9. #9
    Full Member
    Join Date
    October 22nd, 2006
    Posts
    200
    Your programer need to recompile the php with "--with-zlib' '--with-zlib-dir=/usr' ".

    Like you, I have not tried many datafeed import tools but I would have thought that their are others with a longer track record than 09-apr-2008 that would work better. As it would appear it can't handle CJ and a special piece of code had to be written for you to import SAS it is not as advanced as it claims to be.

    My own software can download feeds from a dozen different networks and it only takes a few minutes to set up new ones. It autodetects zip, gz and text files with tab, pipe, comma and semicolon separators. But as you have said it falls short of meeting your needs. My script is mainly designed to get products into a database. I only provide the simplest of tools for the display.

    As I said off the forum I am prepared to allow you a 30 day trial so that you can assess how far short it falls from your needs.

    Bob

  10. #10
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    > "I would have thought that their are others with a longer track record than 09-apr-2008 that would work better." <

    Yes, I thought so, too -- until I tried to buy one.

    Note that I can't "recompile" the Datafeed Studio code, as it is encoded with IonCube to prevent source-code access to the core product.

    Also, Bob, could you clarify -- can your script currently import CJ datafeed files from the CJ server into the product database?

  11. #11
    Full Member
    Join Date
    October 22nd, 2006
    Posts
    200
    I have only accessed the CJ feeds using an http download from CJ UK but as I said it only takes a few minutes to add new networks and change existing ones.

    Bob

  12. #12
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    I haven't checked in a while. Do you still have to pay for CJ feeds?


  13. #13
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Quote Originally Posted by bumpaw
    > I haven't checked in a while. Do you still have to pay for CJ feeds? <
    There was a recent thread noting that they are not currently charging the fee. They didn't ask me for any money.

  14. #14
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    Quote Originally Posted by markwelch
    There was a recent thread noting that they are not currently charging the fee.
    I missed that one. Thanks


  15. #15
    Full Member iolaire's Avatar
    Join Date
    October 3rd, 2006
    Location
    Arlington, VA
    Posts
    229
    Quote Originally Posted by Donk
    Your programer need to recompile the php with "--with-zlib' '--with-zlib-dir=/usr' ". ... Bob
    What bob is trying to say is that your contractor needs to do this on your VPS's php install or maybe all you need to do is uncomment "extension=zlib.so" in the VPS's php.ini in order to be able to work with zip files.

    Maybe you could figure out/learn how to script the ftp/zip steps in LAMP to get all the files split out in individual local files for your developer to use? I believe that the base PHP code is available over on the Data Feed thread. That way you would be responsible for dealing with one limited area - that needs to be customized for every affiliate program.

    Otherwise you will find with each program you need new specs and new development, in addition to the general customization for the various different feed formats.

  16. #16
    Member
    Join Date
    January 18th, 2005
    Posts
    144
    Did you ask CJ if they would give you individual files for each merchant? I think they have this capability. Your next problem will be that some merchants have multiple data files within thier datafeed.

  17. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Accessing ShareASale FTP Datafeeds
    By markwelch in forum ShareASale - SAS
    Replies: 2
    Last Post: February 9th, 2007, 10:22 PM
  2. Accessing Certain Board Features
    By Kellie aka Ms. B in forum Midnight Cafe'
    Replies: 22
    Last Post: December 22nd, 2005, 11:27 AM
  3. HOw many posts until accessing PM's
    By eljefe3 in forum Virtual Family and Off-Topic
    Replies: 7
    Last Post: July 8th, 2005, 10:25 AM
  4. Reports - Anyone having trouble Accessing??
    By JadaKiss in forum Rakuten LinkShare - LS
    Replies: 17
    Last Post: March 13th, 2002, 09:15 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •