[moonshieldmedia]

Hello,

I was wondering how often one is supposed to update the datafeed. Is it every day, every week, every month?

[SSanf]

Frankly if you have lots of merchants quarterly should be enough. Heck, even clothing and gardening supplies should only change once each season. If the feed goes way out of date any more often than that, they better pay a darned big commission or have a whole lot of sales to make it worth the time and effort.

Merchants who think they are worth daily or weekly attention are full of beans unless they are one darned big producer and have an absolutely flawless feed. You need to determine how much time the update takes and what your time is worth.

Every time you update the feed of a merchant you already have on your sites is one new merchant that you DIDN'T have time to get up. There is an opportunity cost to updating merchants. If you update a merchant 5 times, you have one merchant. If you don't update and instead add four merchants, you now have 5 merchants. And, THAT is really worth thinking about. A merchant who is too time demanding can keep you from growth and keep you stuck at the level you already have achieved.

[Mr. Sal]

Originally Posted by SSanf

Frankly if you have lots of merchants quarterly should be enough. Heck, even clothing and gardening supplies should only change once each season. If the feed goes way out of date any more often than that, they better pay a darned big commission or have a whole lot of sales to make it worth the time and effort.

Merchants who think they are worth daily or weekly attention are full of beans unless they are one darned big producer and have an absolutely flawless feed. You need to determine how much time the update takes and what your time is worth.

[MichaelColey]

My updates are all totally automated, so I update as frequently as the merchant updates their datafeed, which is usually daily. I'd rather have accurate information. I actually have most of my datafeed processing routines checking for updates at least once an hour. As soon as a new/changed datafeed appears, it's downloaded and processed.

If you don't list prices, or if you work with merchants whose prices seldom change, it's much less urgent.

Automation is your friend.

[popdawg]

Daily for those that update it often and regularly have sales, weekly for those that keep their feeds fresh and for the rest it depends on their update frequency. I show prices though.
Like Michael, mine are automated.

[SSanf]

I don't show prices. So, the main advantage is getting rid of discontinued items and broken images. If the main thing in their new feed is that they now have 14 gizmos and 16 dowhichies and I already display 12 gizmos and 13 dowhichies, there is a very small return to me for updating the feed.

[Snib]

Originally Posted by MichaelColey

My updates are all totally automated, so I update as frequently as the merchant updates their datafeed, which is usually daily. I'd rather have accurate information. I actually have most of my datafeed processing routines checking for updates at least once an hour. As soon as a new/changed datafeed appears, it's downloaded and processed.

If you don't list prices, or if you work with merchants whose prices seldom change, it's much less urgent.

Automation is your friend.

Wow, now you've raised the bar for me! I need to start checking modified dates. How do you deal with LinkShare's funky FTP system?

- Scott

[MichaelColey]

Originally Posted by Snib

How do you deal with LinkShare's funky FTP system?

With a short Perl program that I wrote:

http://forum.abestweb.com/showthread...5519#post56507

Running the script regularly will basically keep a current local copy of all the Linkshare datafeeds. Linkshare changes the timestamp on the .lmp files when new files are available, so I only download when the timestamp changes.

[john9245]

I try and update my datafeeds on a weekly basis as my scripts for adding the datafeeds to my Database is not 100% automated. They are about 95% automated, which means that I have to do a little bit of work each time I update a new merchants feed within my database.

But I am new to this, so I will keep improving the scripts and hopefully then it will become 100% automated and datafeeds will then be updated as soon as the merchants update their datafeeds.

[Leader]

For my WebMerged stuff, I update once every time H freezes Well, that's an exaggeration, but it doesn't happen much. I hate doing updates and would much rather put on new merchants instead. There has to be a really compelling reason--I don't do w*rk "just for."

For my more automated sites, I'm like John. Updating a site weekly is a much faster and easier proposition (and therefore not nearly as distateful) when it only takes 10 minutes or less to do it! I literally have the game minimized and waiting for my return...don't even have to hit Save!

As for *total* automation, that sounds great in a way, but there are issues I am leery about. I'd have to figure out a way to not be scr*wed if something didn't run right...or if the feed didn't come in (or was corrupted), etc. I'd hate to wake up to a sitefull of blank pages or gibberish!

[FourthWorld]

Originally Posted by Leader

For my WebMerged stuff, I update once every time H freezes Well, that's an exaggeration, but it doesn't happen much. I hate doing updates and would much rather put on new merchants instead.

Leader, would it help smooth out your workflow if I added a feature to WebMerge which would keep track of records and only generate/upload pages for records that have changed?

I've been doing some research on efficient methods of storing this sort of info and I think I may be able to put that in v3.0...

[Leader]

Leader, would it help smooth out your workflow if I added a feature to WebMerge which would keep track of records and only generate/upload pages for records that have changed?

It'd be cool and certainly lessen my annoyance factor (I HATE doing something only to find out it wasn't really necessary!), but I'm not sure how much real w*rk it'd actually save. Once I go through all the steps to be ready to run, it's pretty much done anyway... so to be a real help, the comparison would have to kick in RIGHT AWAY before I did anything else.

As for just working off of "if a record has changed," that wouldn't be specific enough to help anything, though. A lot of times, a merchant's entire feed will get a new "lastupdated" date--which would trigger off as a change--when only a couple of records have changed! (I speculate that every time a merchant uploads something to the system, it stamps it with a new date. So if they reupload the whole feed, it all gets the new date even if only one little thing is what changed!)

It may help if there was an alert that said WHAT in the feed had changed, rather than just going on if "any" change happened (Did a particular field's data change? Which field [Field name/number]? Were records added or deleted? How many? That sort of thing). That way I could decide whether I thought the change was actually important, and proceed accordingly.

[FourthWorld]

Originally Posted by Leader

As for just working off of "if a record has changed," that wouldn't be specific enough to help anything, though. A lot of times, a merchant's entire feed will get a new "lastupdated" date--which would trigger off as a change--when only a couple of records have changed! (I speculate that every time a merchant uploads something to the system, it stamps it with a new date. So if they reupload the whole feed, it all gets the new date even if only one little thing is what changed!)

It may help if there was an alert that said WHAT in the feed had changed, rather than just going on if "any" change happened (Did a particular field's data change? Which field [Field name/number]? Were records added or deleted? How many? That sort of thing). That way I could decide whether I thought the change was actually important, and proceed accordingly.

Hmmmm....interesting. These date stamps are added to each record? Yes, that would make it difficult to determine which ones have changed.

But I like the suggestion of supporting an option to use a key field instead. I'll look into this further to see what I can come up with.

Thanks for the feedback.