Results 1 to 4 of 4
  1. #1
    Join Date
    July 26th, 2011
    Angry Datafeed Duplicates
    Hi all,

    I'm having an issue with datafeeds and duplicates. I've got a datafeed that says it contains about 250,000 products but when scrubbed for duplicates it only has about 20% of that.

    Is that normal or am I dealing with a poorly setup datafeed?

  2. #2
    Join Date
    January 18th, 2005
    It depends on how you define "duplicates."

    Some datafeeds include multiple entries for items that are identical except for size or color. This can be a huge problem, but listing each size or color actually makes sense if the merchant only offers a limited selection of sizes or colors (e.g. we have New Balance 405 shoes, but only in white, and only in sizes 7.5 and 10.5).

    Some datafeeds assign only a single category per item, and thus repeat items which fit into multiple categories (once for each category that applies).

    Another problem: items which are replicated with different imprints (sports teams, colleges, etc.), so that instead of one sports jacket, the SKUs just explode (x5 sizes x 6 colors x 32 sports teams).

    Note, too, that there are some other complicating factors: for example, eBooks might be offered in several different formats (Kindle, ePub, PDF, etc.) which are assigned separate SKUs (sometimes unique ISBNs, but often not). For computer software, I've seen datafeeds that list 5, 10, or 30 SKUs for each software title, because they include regular retail package, downloadables, upgrades, multi-user licenses, etc. all as separate SKUs.

    Making matters worse, some merchants don't include the unique elements of the item in the product title, so there are many identical titles, with the unique elements identified only in the product description.

    Some merchants (or the OPM) can provide datafeeds in different configurations, for publishers who find the default datafeed inconvenient. Before spending time writing a special script to de-dupe the data, always ask whether this is an option.

  3. #3
    Join Date
    July 26th, 2011
    Thank you for the insight, that will greatly help me moving forward.

    I've had a chance to hear back from the retailer providing the datafeed and they are suggesting that the issue lies in their use of wholesalers. They source the same product from different wholesalers but assign them different skus.

    This seems strange but I guess not much stranger than some of the other scenarios you outlined.

    Thanks again!

  4. #4
    Full Member Tech Evangelist's Avatar
    Join Date
    March 16th, 2005
    Mesa, AZ
    That is a strange way to manage SKUs.

    If they are sourcing the same product from 5 different wholesalers, do they have 5 separate product pages on their web site for the same product?

    I frequently see 5 or 10 variations of a product in a data feed that each use the same description. Sometimes the variations are size or color. Because each product is different it would be legitimate for each to have a differnt SKU.
    There's good, fast and cheap. Pick any two.
    [url=]Phoenix SEO[/url] :: [url=]Affiliate Marketing Tutorials[/url]

  5. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Duplicates in one field...
    By lulav in forum WebMerge (
    Replies: 1
    Last Post: November 28th, 2009, 11:37 PM
  2. Replies: 4
    Last Post: June 23rd, 2009, 06:39 PM
  3. PSC making TONS of duplicates?
    By writerguy in forum BettyMills
    Replies: 2
    Last Post: January 15th, 2004, 08:45 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts