Results 1 to 14 of 14
  1. #1
    Schlaumeier cumbrowski's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    640
    Merchant Product Datafeeds for Affilates - 101
    1. Format
    2. Delivery Method
    3. The Information in the feed/Content
    4. Notes
    5. "Appendix"

    Introduction
    I dealt with feeds from hundreds of merchants, ALL US Affiliate Networks. All kind of formats and delivery methods and also implemented some Datafeeds for Merchants (as AM) as well.

    I believe I know what I am talking about so I took the time to take all my notes and individual emails regarding this topic and put them together into this one (long) document.

    1. Format
    Format. Delimited (comma, semicolon, tab, pipe, tilde whatever. Tab makes a lot of sense since it can't be used on the WEB and should be replaced in the content anyway.
    Row delimiter should be a windows <cr><lf>. No Text Qualifier like ".

    Provide Column Headers in the first Row. NO SPECIAL Characters. Use only 0-9,a-z,A-Z and "-" or "_" if you have to. No !,?,&,%,#,',",(,),\,/ etc. Those are reserved characters in applications or programming languages a lot of times. Also avoid blanks.

    Delimited Files MUST HAVE the same number of columns for every single record.

    If an information is not available for a product send the initial value instead (0,nothing,1/1/1900 etc.).


    Yes, this one is directed straight at Linkshare. No Column Headers to begin with*,
    First row with 4 columns ("HDR", MID, Merchant Name, Date last updated) and last row with 2 columns ("TRL", number of products) is against the definition of delimited.

    Btw. thanks to let me know at the end how much products I just read in sequential order. My Computer can not count .


    XML is fine too IF you provide the Data via a Webservice and if you can generate correct XML (most can't).
    No XML if you provide the Feed as a File.


    Gzip it if necessary (large feeds)

    2. Delivery Method
    FTP Download or available via HTTP on the Site (if the feed is < 5 MB), Web Service (SOAP/XML).
    No FTP Upload (very kind if a merchant offers that, but most of the time impractical for affiliates).
    Fine if it is an option, but it should not be the only option.


    If you make it available as file don't make the name dynamic (make the creation date part of the file name for example).
    You might do that if:
    - The File is provided via FTP (NOT Http) that affiliates can write a script checking for the file(s) available.
    - The File is scheduled to be generated automatically - A RELIABLY SCHEDULE, daily, weekly, monthly whatever.


    If you want to provide affiliates the information that the feed changed for an HTTP Feed that they can check first, if something changed before they waste the time and resources to download and process the feed.
    Have a second file with a fix filename available via HTTP also that contains the information like: Date/Time the feed was updated the last time. Filename of current feed, size, number of records etc.


    Affiliates have different levels of sophistication and available tools.
    Some build links to selected products manually, some use simple scripts or software and others have a high-end enterprise server at their disposal.
    Some run their stuff under windows, others use Linux.
    The more formats and delivery methods provided, the better and the more affiliates can and will use the feed.


    Documentation is always king. A description of the feed structure and format takes the guess work out of the work with the feed for the affiliate.

    3. The Information in the feed/Content
    Underlined Columns/Attributes are considered "must have" by most affiliates.

    - SKU/Product ID

    A unique ID for a product in the merchant catalog.
    Either the internal product id or SKU or UPC/EAN/ISBN Number of the Item (which is also unique).

    No duplicates in the feed. If alternative categories are available for the product use another column for the alternative categorie(s).

    If a product has multiple size and color options, don't send a record for each possible combination unless the merchant has a separate product detail page for each of them. Provide the available options in a separate column. Add the main SKU only.



    - Name of the Product
    - Description of the Product

    Short Description, Long Description (or both in separate columns).
    Additional Information and Features may be as separate fields, depends on how the merchant has it.

    No HTML Code unless it makes sense, but keep it simple and don't use formatting like font tags or colors etc.

    Tables where necessary, bold, underline, unsorted or sorted lists, that's it.

    - Price of the Product the customer has to pay
    - Product URL

    Coded Link (URL) to the Product Detail Page with Affiliate ID

    - URL to Product Detail Image
    - URL to Product Thumbnail Image
    - Regular Price or Retail Price (might be the same as "Price").
    - Product Category

    "Jewelry" for example or if it is in a subcategory send it delimited by ">" or any other unique delimiter that will never be part of a category name, like this "Jewelry>Rings>Engagement Rings"

    - ISBN Number (Books)
    - UPC (A) Number
    - International EAN
    - Manufacturer Name / Brand Name (separate Column if there is a difference)
    - Manufacturer Part Number
    - Color Properties delimited
    - Size Properties delimited
    - Keyword Suggestions
    - Artist (Music, Art)
    - Author (Books)
    - Publisher / Label / Studio (Books / Music / DVD)
    - Shipping Information (available Options like Ground/Next Day, restrictions)
    - Shipping Cost for a single Unit (if flat across the country)
    - Availability

    Available, ships within 24 h, back order, temporary out of stock etc. Don't add products to the feed that are out of stock for good.
    Only add what you have or what you can get)

    - Related Items

    Could be a list of SKU's in delimited format like "123|321|456"

    - Flags like Best Seller, High Rating, Clearance, On Sale etc.
    - Link to Product Category
    - Buy URL that add the item right to the Basket
    - Brand/Manufacturer Logo (URL to Image)

    4. Notes
    The more the better. Not all affiliates use feeds the same way. The more information are provided by the merchant the more can affiliates do with them, its as simple as that.

    Some use only selected products others use simple Showcase Creators, price drops, on sale features, price comparison, niche site where people look for very specific things etc. Some use it for multiple of the mentioned purposes.
    Same here, the more you provide, the more affiliates can and will use the feed.

    Documentation is still king. Provide Information about the content. What can affiliates expect to find in the columns/attributes.This becomes especially important for Network Feeds where the Feed Structure is fix (CJ) or only provides the ability to use non-descriptive custom columns (ShareASale).

    If you use Standard Columns of a Network Feed for something else, fine, but document it (that's true for all Network Feeds from CJ, BeFree,Linkshare,ShareASale, Performics etc.).

    Pick at least Columns where the intended Data would have the same format as the Data you dump in there, for example don't dump a character string in a column that was meant to hold dates or numbers or currency, some Networks sadly allow that.

    If you use a column/attribute of a standard network feed, always send data and not leave it empty for products where you don't have the information.
    For example. If you provide a "Sale Price", send at least 0 or the "Regular Price" for products that are not on sale and don't leave it empty (null).
    The technical term for that is "Initialize" the Columns/Attributes used.

    I can live with feeds that don't have enough information. I simply don't use them. I can fill some gaps myself for some and do it, if I believe that it is worthwhile.

    What I can not live with are errors in the file structure or garbage in the content. If " is not specified as text qualifier, don't enclose strings with it. If it is specified, replace " in the content with "" (2x ").

    Use only ASCII code 32 (space) to 126 (tilde ~) for the content (called 7 bit ASCII, US-Charset, ISO 636 or ANSI (American National Standards Institute) Standard x3.4).

    I waste most of my time today on: fixing datafeed structures (pre-processing), validating technically and often "logically" the content of each column.

    I use provided Category structures. I have to spend a lot of time with messed up category structures such as:

    - thousands of products assigned to a single sub category
    - category with a Subcategory with the same name
    - subcategory with another single sub category only, but no products
    - categories with hundreds of sub categories etc.

    All things that make the "Catalog" not "browsable" by a user.

    I think that wraps it up. This text is a summary of various emails I sent to Merchants and Networks.

    Some to complain about issues with the feed others to make recommendations and also emails to merchants that started with an affiliate program and are thankful for every information that helps them to get their program going (I like those the most since they are usually the ones that really listen and try as much as they can to help you out, what is ironically a 2 way street since I on the other hand can do a better job promoting them and generate sales for them and commission for me).

    5. "Appendix"

    * Note to the Linkshare Feed.
    Here is the Column Structure as far as I understand.
    PRODUCT_ID
    NAME
    SKU_NUMBER
    PRIMARY_CATEGORY
    SECONDARY_CATEGORY (can be multiple separated by ~~)
    PRODUCT_URL
    PRODUCT_IMAGE_URL
    PRODUCT_BUY_URL (adds product to basket, at least that's what it is meant for)
    SHORT_DESCRIPTION
    LONG_DESCRIPTION
    DISCOUNT_VALUE
    DISCOUNT_TYPE (AMOUNT, PERCENTAGE)
    SALE_PRICE
    RETAIL_PRICE
    DISCOUNT_BEGIN_DATE
    DISCOUNT_END_DATE
    BRAND
    SHIPPING_COST
    KEYWORDS
    MANUFACTURER_PART_NUMBER
    MANUFACTURER_NAME
    SHIPPING_INFORMATION
    AVAILABILITY
    UPC
    CLASS_ID (**)
    CURRENCY
    M1 (still not sure what that one is for)
    PIXEL_TRACKING

    ** Standard # of Columns per Product is 28 for the delimited feed.
    Merchants can add Custom "Columns". If a Class_ID is provided chances are good (you can't be 100% sure) at the feed has 38 Columns.

    The Structure of the 10 additional columns depend on the "Class ID" value. A merchant can use multiple classes and/or no class depending on his needs.
    That's the merchant feeds where it shows "yes" for Attribute File in the LS Merchandiser List.

    Linkshare's standard format for feeds is XML (although they don't get the XML one right either). The Class ID comes from the XML Definition of the Feed,
    Delimited Feeds are converted XML Feeds. Watch for errors. I had feeds where the merchant used 2 different classes and no classes. If the first products have no class, LS starts creating the feed using 28 Columns only and extends it to 38 later when it realizes that Attributes are used in the Feed.

    That made my import program die. I pre-process the merchants feed now since they were not able to fix that last year. Haven't got any news from them that they did fix it.

    AT least they fixed some other "bugs" which I could not fix with pre-processing, but I don't want to go into further details with that.
    Carsten Cumbrowski
    Marketer & Entrepreneur @ Cumbrowski.com
    Free & no BS Internet Marketing Resources

  2. #2
    Affiliate Manager nish's Avatar
    Join Date
    July 6th, 2005
    Posts
    762
    Carsten,

    Excellent post.

    No duplicates in the feed. If alternative categories are available for the product use another column for the alternative categorie(s).
    If a product can be under multiple categories, would it be easier to make both the product-id and the category as the primary keys and have multiple rows?

    -nishith

  3. #3
    Schlaumeier cumbrowski's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    640
    Quote Originally Posted by nish
    Carsten,

    Excellent post.



    If a product can be under multiple categories, would it be easier to make both the product-id and the category as the primary keys and have multiple rows?

    -nishith
    No, because how would you be able to distinguish between the same product that is attached to multiple categories. You end up littering your Database with duplicate data (a lot of data) and if you think about making it a primary key that uses to columns, don't! Your DB Performance goes to hell.
    If you are using MS SQL Server, check following site http://www.SQLServerCentral.com. Good Site, tons of free resources.

    The multi-category and multi color/size combination issue is actually the only PRO I can think of when it comes to the choice between FILE feeds in XML format vs. delimited textfile (zipped or not zipped). Delimited Files are only 2 dimensional.

    Adding another dimiension for a property requires the creation of delimited content within the delimited file. XML has no problem with that.

    But XML Files are much bigger and still not always as easy to process (especially if the XML Format is proprietary and you don't get the XML Schema send with the XML Data file).

    Another benefit of XML is the amount of Data per "Record" or Product you can send. XML has what? 2 GB limit or something like that.
    Different with delimited files. The maximum number of characters per "Line" is usuallly limited by the software that's using them. 4 KB, 8 KB etc. but we rarely have an issue with that when we deal with product data feeds.

    XML makes perfect sense for a Webservice using SOAP.
    Tools today that support the creation and/or use of Webservices are getting easier to use every day. You don't have to deal with the actual XML most of the time anymore.

    The conversion to XML, transfer, receival and conversion back , to whatever it is you process at the receiving end, happens often in the background automatically without you having to worry about it.

    A good analogy would be TCP/IP. You don't have to understand the structure of the traffic via TCP/IP if you do a FTP Transfer. It's used by Server and Client in the background automatically without you having to worry about it.
    Carsten Cumbrowski
    Marketer & Entrepreneur @ Cumbrowski.com
    Free & no BS Internet Marketing Resources

  4. #4
    Affiliate Manager nish's Avatar
    Join Date
    July 6th, 2005
    Posts
    762
    Quote Originally Posted by cumbrowski
    Adding another dimiension for a property requires the creation of delimited content within the delimited file.
    So within a tab delimited file, what do you propose in order to allow multiple categories.
    1. A column for each category (& throw in a limit, lets say 3, to the max number of categories that a product can have)
    2. comma or pipe delimited category column.

    Which of these would be appropriate?

    Thanks,
    -Nishith

  5. #5
    Schlaumeier cumbrowski's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    640
    Quote Originally Posted by nish
    So within a tab delimited file, what do you propose in order to allow multiple categories.
    1. A column for each category (& throw in a limit, lets say 3, to the max number of categories that a product can have)
    2. comma or pipe delimited category column.

    Which of these would be appropriate?

    Thanks,
    -Nishith
    Option A.
    1. addidional Columns "Alternative Category" and "2nd Alternative Category"
    2. Should stick to the separator proposed for the Main Category, which is ">"
    such as "Clearance>Widget Clearance"

    Option B.
    1. Column "Alternative Categories"
    2. Subcategories separated by ">" as in Option A.
    Separator for multiple alternative Categories could be "~"

    Example: "Clearance>Widget Clearance~Holiday Features>Widgets"

    Option B is moe flexible, but harder to build by the merchant and harder to use by the affiliate. I think that multiple alternative categories are rare. It's usually a featured category that is the alternative one.

    If there is another one or even more, then there would be a custom solution in order. Ususually if your categories are so generic that a single product would match 3 or more, a "smart" or "guided" navigation should be implemented. Which means that a User can "narror" or filter the product listings by selecting specific product properties like "Brand" or "Price Range", "Size", "Color" etc. That would be a much better way to build a catalog anyway, but requires that the merchant provides all the product properties in individual columns in the data feed.
    Carsten Cumbrowski
    Marketer & Entrepreneur @ Cumbrowski.com
    Free & no BS Internet Marketing Resources

  6. #6
    Affiliate Manager nish's Avatar
    Join Date
    July 6th, 2005
    Posts
    762
    Carsten,

    In option B, are you suggesting to have two columns: Category and Alternative-Categories? Isn't it better to have just one Category column and then as you are suggesting, seperate multiple categories using '~' ?

    I'm assuming that it won't be a big pain for affiliates to explode/parse such a column. Please correct me if you think otherwise.

    cheers!
    -nishith

  7. #7
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Great post, Carsten. I hear you loud and clear on XML.

    There's an effort underway from some of the regulars here to define a single datafeed standard:
    http://forum.abestweb.com/showthread...&highlight=xml

    It might be good to see if we can combine the best of your ideas with the groundwork done there thus far to make sure the final spec serves everyone's needs well.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  8. #8
    Schlaumeier cumbrowski's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    640
    Quote Originally Posted by nish
    Carsten,

    In option B, are you suggesting to have two columns: Category and Alternative-Categories? Isn't it better to have just one Category column and then as you are suggesting, seperate multiple categories using '~' ?

    I'm assuming that it won't be a big pain for affiliates to explode/parse such a column. Please correct me if you think otherwise.

    cheers!
    -nishith
    Both of your options would work.

    Using 2 columns would make it a bit easier for Affiliates that are not very technical to have at least something to work with.

    If you don't, fine, but you might get more emails asking for help from your affiliates
    Carsten Cumbrowski
    Marketer & Entrepreneur @ Cumbrowski.com
    Free & no BS Internet Marketing Resources

  9. #9
    Schlaumeier cumbrowski's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    640
    Quote Originally Posted by FourthWorld
    Great post, Carsten. I hear you loud and clear on XML.

    There's an effort underway from some of the regulars here to define a single datafeed standard:
    http://forum.abestweb.com/showthread...&highlight=xml

    It might be good to see if we can combine the best of your ideas with the groundwork done there thus far to make sure the final spec serves everyone's needs well.
    I saw that thread and posted there the link to this thread http://forum.abestweb.com/showthread...xml#post523968.
    I created a new thread because I didn't wanted it to get "under" in the 100's of Posts in the "Merchant Datafeeds.. WHY???" thread.

    I am all for a standard but would go further than that. I would create sub standards for the various types of feeds for different purposes (e.g. standard for price comparison feed, showcase creator feed, browsable catalog feed , deals and savings feeds etc. ). Based on the purpose become certain fields in the feed required and others optional.

    The standard should also explain for each defined column (required and optional ones) what the purpose of the information is and what affiliates usually do with it.

    The standard should also include specifications about the format of the feed, I just say "initialization" of empty columns, allowed and forbidden characters in content etc.

    At the very end is it very important to get ALL of the Networks (at least CJ, LS, SAS and Performics) to participate for comments, recommendations and much more important to make the commitment to provide feeds following the standard to affiliates in their network. Standards are useless, if the majority does not care and follows it. The Standard would remain a suggestion or idea and nothing more.
    Carsten Cumbrowski
    Marketer & Entrepreneur @ Cumbrowski.com
    Free & no BS Internet Marketing Resources

  10. #10
    Affiliate Manager nish's Avatar
    Join Date
    July 6th, 2005
    Posts
    762
    Quote Originally Posted by cumbrowski
    Both of your options would work.

    Using 2 columns would make it a bit easier for Affiliates that are not very technical to have at least something to work with.

    If you don't, fine, but you might get more emails asking for help from your affiliates
    Tough choice

    Thanks,
    -nishith

  11. #11
    Online Marketing Consultant
    Join Date
    October 16th, 2006
    Location
    Washington D.C.
    Posts
    2,195
    I have a question and think it might fit well here. I have an affiliate who wants to use our xml data dump but the affiliate ids are not in it. Is there a way for her to easily apply hey PIDs to the feed when she gets the data dump? I am not a techy by any means so please speak in very very easy to understand lamens terms. Thanks.
    Adam Riemer Marketing, LLC. Online Marketing Blog and Affiliate Management Company
    Do you need help with your Marketing or Sales funneling, write me at adamr (at) adamriemer (dot) me

  12. #12
    Schlaumeier cumbrowski's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    640
    yep, urls in the feeds are just like that. Sometimes they have a "place holder" such as "[AffiliateID]" or something like that as part of the link, which you only have to replace with your affiliate ID and that's it.
    Carsten Cumbrowski
    Marketer & Entrepreneur @ Cumbrowski.com
    Free & no BS Internet Marketing Resources

  13. #13
    Newbie
    Join Date
    October 21st, 2007
    Posts
    2
    Hi Cumbrowski,

    Excellent posts. I am already very impressed with your own site which led me to many affiliate programs and links to the documentation. I was quite successful with the CJ's webservice document.

    I was working on integrated the linkshare webservice and i am not able to use the method exposed through the service TextPromotion?wsdl of linkshare

    This web services exposes the following methods

    GetAllLinks ( )
    GetMerchantLinks ( )
    GetNetworkLinks ( )

    But i am not able to access the method. Is there anyway you can provide me direction on how to achieve this. I am using asp.net

    Here is my current scripts

    Sub performProductSearch(ByVal myLicenseKey As String)

    Dim productSearchService As TextPromotion.GetAllLinksRequest = New TextPromotion.GetAllLinksRequest()
    Dim productResult As TextPromotion.GetAllLinksResponse = New TextPromotion.GetAllLinksResponse
    Dim j As Integer

    productResult = productSearchService.GetAllLinks(myLicenseKey)
    Response.Write("<B>Product Search Results</B><BR><BR>")

    For j = 0 To productResult.products.Length - 1

    Response.Write(productResult.products(j).name & "<BR>")
    Response.Write(productResult.products(j).price & "<BR>")
    Response.Write(productResult.products(j).buyUrl & "<BR>")
    Response.Write("<BR>___________________________________________________________________<BR><BR>")

    Thanks in advance
    Next
    End Sub

  14. #14
    Schlaumeier cumbrowski's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    640
    Quote Originally Posted by TECH_NET
    Hi Cumbrowski,

    Excellent posts. I am already very impressed with your own site which led me to many affiliate programs and links to the documentation. I was quite successful with the CJ's webservice document.

    I was working on integrated the linkshare webservice and i am not able to use the method exposed through the service TextPromotion?wsdl of linkshare

    This web services exposes the following methods

    GetAllLinks ( )
    GetMerchantLinks ( )
    GetNetworkLinks ( )

    But i am not able to access the method. Is there anyway you can provide me direction on how to achieve this. I am using asp.net

    Here is my current scripts

    Sub performProductSearch(ByVal myLicenseKey As String)

    Dim productSearchService As TextPromotion.GetAllLinksRequest = New TextPromotion.GetAllLinksRequest()
    Dim productResult As TextPromotion.GetAllLinksResponse = New TextPromotion.GetAllLinksResponse
    Dim j As Integer

    productResult = productSearchService.GetAllLinks(myLicenseKey)
    Response.Write("<B>Product Search Results</B><BR><BR>")

    For j = 0 To productResult.products.Length - 1

    Response.Write(productResult.products(j).name & "<BR>")
    Response.Write(productResult.products(j).price & "<BR>")
    Response.Write(productResult.products(j).buyUrl & "<BR>")
    Response.Write("<BR>___________________________________________________________________<BR><BR>")

    Thanks in advance
    Next
    End Sub
    Hi Tech Net,

    I am unfortunately not the .NET crack myself, something that is on my to-do list, but I wanted to get out of the programming business anyway (I am not a programmer at heart, only good at it, if I do it. That's at least what I got told).

    Anyhow, what is the definition of "TextPromotion"? Is it a SOAP Web Services object? What are you getting back? Anything? Any errors?

    Note that the Linkshare WS requires SSL and the permissions work different than with CJ's web services.

    Linkshare
    Code:
    
    




    versus CJ which does not require SSL and has all tokens and IDs transmitted with the SOAP request in the XML.
    Also, did you get your account enabled for the web services? You have to contact LinkLocatorDirect at linkshare dot com to get your account enabled.

    I would check those things first.
    Carsten Cumbrowski
    Marketer & Entrepreneur @ Cumbrowski.com
    Free & no BS Internet Marketing Resources

  15. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. product datafeeds
    By hmc in forum Rakuten LinkShare - LS
    Replies: 0
    Last Post: March 22nd, 2006, 03:50 AM
  2. Single Product Datafeeds?
    By Cagles Mill in forum ShareASale - SAS
    Replies: 2
    Last Post: March 27th, 2004, 11:30 AM
  3. Datafeeds 101 - A Beginner's Primer
    By Linda - 5starAffiliatePrograms in forum Programming / Datafeeds / Tools
    Replies: 0
    Last Post: February 17th, 2004, 10:51 AM
  4. DataFeeds 101
    By bcorso in forum Programming / Datafeeds / Tools
    Replies: 1
    Last Post: November 24th, 2003, 09:28 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •