Results 1 to 14 of 14
November 3rd, 2010, 07:36 PM #1Datafeed Category Problems
After reading some of the threads, I don't think I'm alone is observing that a Large(?) number for datafeeds don't have MERCHANT Categories (the have SAS catagories, but that doesn't help). Most "store-types" would really bendfit from at least a 1 level category break-down. I have the programming chops to do it, but with 5 out of 5 that I've looked at, there's nothing there!
A) Same experience for others?
B) Is there a way to tell if they have this column in the .csv?
C) Other than just randon display of 1-12 at a time, with a search field, what other approaches have you tried?
D) Does contacting the merchant and requesting that they add these columns help?
November 3rd, 2010, 09:26 PM #2
Yup, datafeed categories are frequently not filled out, they taxonomy nodes aren't normalized across merchant datafeed files, they have different depth levels, and the formatting of the parent child node categories isn't standardized. (e.g. Parent ->Child or Parent, Child or Parent:Child or Parent~Child, etc)
I've often thought about hiring low cost labor to unify & normalize the taxonomies across all merchant programs, but the benefit vs cost doesn't quite add up. It would be a continuous, ongoing effort as these taxonomies are very dynamic.
I once, about 6 years ago, took a snapshot of Amazon's taxonomy, then a year later, tried to reconcile (e.g. merge) that taxonomy with my year old snapshot. It was an impossible task.
There has been a lot of CS research on automating the normalization of taxonomies from different sources. They even have their own conference each year. I've written an interactive program to merge similar taxonomy nodes. It works well, but is super boring, so I never finished running it. I've got 800 programs that I was trying to unify though, including Amazon. A big task / huge task.
If you ever figure out a cool algorithm you should update this thread, I find the subject pretty fascinating in a geeky sort of way.
Last edited by isellstuff; November 3rd, 2010 at 09:28 PM.Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!
November 3rd, 2010, 09:54 PM #3Thanks
Thanks. Very insightful. At least I know I didn't miss something obvious. It's a shame, because irregularities with categories pretty much kills a lot of obvious presentation/organization things we could all do. S,o thanks for saving me the grief.
We struggle with categorization of items even in our original source (non-affiliate) stores. Seems that in the minds of some people (my wife!) any given item could be put in one or more categories that don't quite make sense to a different person (me!). It can be very subjective. Then get all that consistent across all products in the store. Very annoying and TIME CONSUMING!
That leads to the next question... do you know of any merchants (I'm in Share-A-Sale) who have done a solid job of presenting categories/subs? Any with "good reputations" for this kind of thing? R
November 3rd, 2010, 10:13 PM #4
November 3rd, 2010, 10:27 PM #5Thanks
Got to hand it to you....I've created 2 threads and you've responded with a plug for check sales both times. Not annoyed at all. Respect for your persistence. At the right time, I'll check your store and feed out. In this day and age of online banking and eft, etc, are people still writing checks? I write very few.
November 3rd, 2010, 10:50 PM #6Got to hand it to you....I've created 2 threads and you've responded with a plug for check sales both times.
November 3rd, 2010, 10:54 PM #7
I, too, write almost no checks. However online check sales are actually a growing business - for several reasons. To illustrate some of the growth, GirlyChecks, which is only one year old right now, went from concept to ShareASale PowerRank 100 in just over 10 months (and is at #61 this evening). Checks are best sold as a niche item based on category of design - and we have checks for almost every niche.
November 3rd, 2010, 11:18 PM #8
It's a shame, because irregularities with categories pretty much kills a lot of obvious presentation/organization things we could all do
- Join Date
- April 6th, 2006
This is one of my biggest peeves about datafeeds, but in my previous career at an investment company, we had two full time database administrators.
While classifying corporate bonds isn't quite the same as merchant categories, the need for clean data spans any field. I've now come to accept it as part of my weekly routine..
Although it still remains my pet peeve
November 4th, 2010, 05:29 AM #9
- Join Date
- March 13th, 2010
I have integrated about 250 merchants in my custom system. I use a "category swap" table. When I signup to a merchant, I go thru (manually read) their datafeed and identify what is the primary category of their range of products.
So, firstly a rule goes in that says put all products into the "primary" category e.g. Clothing.
Then, rules are setup based on unique Keywords in the Product Name.
UniqueKeyword => MatchingCategory ... ex. Shoes, Sneakers, Clogs, Boots go to the "Shoes / Boots" category. Then the script categorizes products based on above rules for this Vendor whenever the datafeed is updated.
This way, what is matched by Keyword goes into the correct category... while the rest stay back in the primary category. This gives a fairly accurate categorization.
Yes, its time-consuming and boring, but have todo this once for each new merchant. Its much more work for the "sell it all" kind of online malls, because you cannnot identify the primary category for those kind of merchants.
I keep rules vendor-specific, that gives the most accurate results.
November 4th, 2010, 09:38 AM #10
Netmktg, pretty interesting. My overall taxonomy goal was to create something like Shopzilla has. I think Shopzilla starts out with a single datafeed format, though... Let's take a search for playstation 3, they automatically kick you to this page, straight from search:
Shopzilla - Playstation3 Playstation 3 Consoles shopping - Video Games online
If you look at the top, they have taxonomy node navigation. e.g.
Home -> Video Games -> Video Game Consoles -> Palystation 3 Consoles
But, I should point out that my website is probably a little larger than Shopzilla's, in that I have more products across a larger range of taxonomy nodes. A well focused niche website will have an easier taxonomy problem.
Taxonomy nodes can and do exist in multiple locations. It has to be that way to help people find things. For example:
Electronics > Video Games -> Hardware -> Consoles -> Xbox 360
Video Games -> Hardware -> Consoles -> Xbox 360
Toys -> Video Games -> Hardware -> Consoles -> Xbox 360
New product categories are created every year. Here are a couple of (simplified) examples.
Apple starts selling iPads
Electronics -> Computers -> Apple -> iPads
Apple ups the memory on a device
Electronics -> MP3 Players -> Apple -> Nano -> 16GB
My decision has been to focus on search. I display some taxonomy nodes, but without unification, its not a great customer experience. BTW, if you track your user interaction on your website, you will probably find that they don't use taxonomy navigation very much, and when they do, they can get frustrated quickly and leave. This is because they have to guess where things are located and frequently guess incorrectly. An extreme case, how successful are you at navigating dmoz or the yahoo directory from the root without the benefit of search?
Of course there are SEO benefits, crawlers like taxonomy structure as long as it is content rich.
Last edited by isellstuff; November 4th, 2010 at 09:40 AM.Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!
November 4th, 2010, 01:17 PM #11
Have any of you looked into eBay's taxonomy? That's probably as comprehensive as any, and I believe they let you use the API even if it's for things besides promoting eBay. There's still the problem of mapping from individual merchants into a generic taxonomy.
Whether the problem is attacked with manpower or technology (or more likely both), it's still a mammoth undertaking.
November 4th, 2010, 01:51 PM #12
- Join Date
- January 18th, 2005
Depending on what types of merchants you are looking for, we manage several programs that have very well categorized datafeeds. Check my signature or email me at affiliates at teamloxly.com for more details on a specific program that you are interested in.Deborah Carney
TeamLoxly.com BookGoodies.com ABCsPlus.com
December 19th, 2010, 06:00 AM #13
- Join Date
- November 3rd, 2009
I built my own system, I manually map each category from a datafeed to one of my own categories.
Despite I've built it so that I can map one or more categories with few clicks, this requires a lot of time ( several hours for each datafeeds, assuming there's a rather large number of categories ), no doubt, but allows a decent categorization, and needs to be done only once per merchant ( you obviously need to check from time to time and match newly created categories ).
Unfortunately some merchants have very bad categorization methods, I've seen categories named "Green", "Orange" etc, with no parent category specified so I actually need to browse the merchant website and see where that category belongs.
Not to mention products added to random not relevant categories, but you can't do anything about this.
I find the problem of missing UPCs to maybe be more problematic that category matching.
July 6th, 2011, 12:23 PM #14
- Join Date
- July 6th, 2011
data feeds come in many different shapes and forms and over the years I have had the same problem with merchants not filling out their category section properly.
I have developed a CMS for data feeds that allows me to add and manage the categories my website requires and the system does a number of searches and processing to each row in the feed to determine what the category is.
So far it has worked out great. It is extremely accurate and no more do I need to manually read the data feeds or custom code things.