Results 1 to 3 of 3
February 14th, 2007, 06:59 PM #1
SAS Datafeed Question: LastUpdated Date Validity?
- Join Date
- January 18th, 2005
This is a question for SAS staff AND for other affiliates who have experience working with SAS datafeeds. My question is not about "how should these things work?" but instead, "Have affiliates had experiences where these things (specifically, the LastUpdated field) don't work the way they are supposed to?"
I'm working on a script to import datafeed files, and right now I'm trying to figure out how little I can do before concluding that a product row HAS or HAS NOT been updated.
Assume that I import a datafeed file (filestamped February 9) that contains two rows indicating a ProductID number (first column) and LastUpdated date (2nd to last column = 18th column):
444444101 . . . 2/6/07 8:39 AM
444444102 . . . 2/9/07 5:42 PM
(1) While loading a new datafeed, my script recognizes that there is already an entry (from a previous file load) for "444444101 . . . 2/6/07 8:39 AM" -- Can I conclude that this record has NOT been updated (e.g. there will never be a row where the price has changed but the update-date has not changed)? Or do I need to conduct additional checks on the other 17 columns to verify "for sure" that no other fields within the record have changed even though the date/time hasn't changed?
(2) Of course, if my script recognizes that there is a prior entry (from a previous file load) for "444444102 2/6/07 8:39 PM" then I am assuming that this record has been modified in some way. Are there situations where "LastUpdated" is changed but no other data in a row has changed?
(3) How frequently do merchant datafeeds trigger "all new product IDs" even though the product names, descriptions, etc. have not changed? That is, how often will I find that ProductID "444444101 . . . 2/6/07 8:39 AM" in one datafeed contains the same product information as ProductID "444455102 . . . 2/9/07 5:42 PM" in a subsequent datafeed that doesn't include any record for Product ID "444444101" ??
My goal is to insure that I add records only when there is a change, without wasting CPU time conducting unnecessary checks. (Whenever there is a change, I will add a new record, not replace the previous record, so that I can track changes and insure that they are applied forward to other tables -- after the entire update cycle is complete, including manual review, the older records for each ProductID will be deleted.)
If there is some uncertainty, then I would go ahead and conduct additional checks to determine if there has been a "real" change to each product record, but I'd certainly prefer to speed up my datafeed-processing time by relying on the update-date to identify modified records.
Last edited by markwelch; February 14th, 2007 at 07:24 PM. Reason: Want to narrow this thread to just "LastUpdated" issues, not other "broken datafeed" problems
February 14th, 2007, 07:18 PM #2Originally Posted by markwelch
Welcome to datafeed hell. Long live GoldenCan.
February 14th, 2007, 08:34 PM #3
Since I don't put in a date from the merchant-side, I assume that SAS adds it. So, it would logically show the last time a feed was uploaded to them. So IF your merchants use logic, and only upload one if there's changes, the date should actually correspond to updates.
As for the product IDs, when I dl'd my feed from the aff side, I saw that SAS's "product IDs" are nothing like what I put in from my end. However, the lack of burning flames from datafeed affs would indicate that these numbers stay the same unless the merchant changes them. SAS must use some kind of formula to arrive at the final product, rather than some randomized output.
That said, in any large pool, there's always going to be someone with a monkey wrench to throw in the works. So you may have the misfortune of hitting a merchant who sends up duplicate feeds just to "make it look newer," or one that changes their product IDs, or both. I would suggest watching what really happens for the first few runs, and if you find any oddballs, you can code up something that'll handle their feeds differently than the rest.
By OldBlue in forum Starting an Affiliate Program & Merchant Q&AReplies: 25Last Post: August 2nd, 2013, 03:07 AM
By stn2001 in forum Commission Junction - CJReplies: 3Last Post: July 4th, 2009, 03:23 PM
By crm911 in forum BettyMillsReplies: 4Last Post: May 30th, 2009, 08:31 AM
By parentsworld in forum Programming / Datafeeds / ToolsReplies: 2Last Post: December 9th, 2002, 12:14 AM