Page 1 of 2 12 LastLast
Results 1 to 25 of 35
  1. #1
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    What percentage of a feed changes, and how often?
    I'm exploring ways to implement an option for WebMerge in which it would only process records that have changed since the last upload.

    So the question for you is two-fold:

    1. When a new feed is delivered, what percentage of records have changed since the previous one?

    2. How often do feeds change on average?

    I know these will vary quite a bit, I'm just looking for broad averages.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  2. #2
    Full Member
    Join Date
    January 18th, 2005
    Posts
    413
    Quote Originally Posted by FourthWorld
    2. How often do feeds change on average?
    Yesterday I received my weekly update of ~50 datafeeds from CJ. The "last update info" for each datafeed is listed below. As can calculate from this list:
    - 40% of the datafeeds were updated within the last week
    - 58% were updated within the last month

    Regards,

    FRANK,
    Baertracks

    12/6/2002
    12/7/2002
    12/7/2002
    2/27/2003
    3/31/2003
    9/15/2003
    9/19/2003
    10/29/2003
    1/1/2004
    4/20/2004
    5/30/2004
    6/3/2004
    7/10/2004
    7/19/2004
    8/25/2004
    10/22/2004
    11/24/2004
    11/25/2004
    12/8/2004
    12/16/2004
    1/26/2005
    1/28/2005
    2/2/2005
    2/8/2005
    2/10/2005
    2/16/2005
    2/19/2005
    2/21/2005
    2/22/2005
    2/22/2005
    2/23/2005
    2/23/2005
    2/23/2005
    2/23/2005
    2/24/2005
    2/25/2005
    2/25/2005
    2/25/2005
    2/25/2005
    2/25/2005
    2/26/2005
    2/26/2005
    2/26/2005
    2/26/2005
    2/26/2005
    2/26/2005
    2/26/2005
    2/26/2005
    2/26/2005

  3. #3
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    The percentage of records can be highly seasonal, some feeds haven't changed in 2 years, others are dropping and adding products daily.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  4. #4
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    It varies. Great answer huh?

    This might work, if you could allow users to select the field to base the update on.

    The real challenge will occur for those of us that modify the feed prior to running WM. This feature would not work at all, or be of limited use.

    What "qualifiers" does everyone have in mind for an update?

    Fred

  5. #5
    Outsourced Program Manager Jorge - SHOPiMAR's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    3,550

    This might work, if you could allow users to select the field to base the update on.
    This would be a good feature.

    Also I think a 30 day window for updates is probably the norm.

  6. #6
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    Quote Originally Posted by buy_online
    The real challenge will occur for those of us that modify the feed prior to running WM. This feature would not work at all, or be of limited use.
    That too. I modify every feed at least a little bit and some a whole lot. Especially the feeds without categories.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  7. #7
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Quote Originally Posted by loxly
    That too. I modify every feed at least a little bit and some a whole lot. Especially the feeds without categories.
    What if you could make a list of categories which get assigned to specific records based on SKU# or other unique identifier? This would in effect let you add a new column to the feed, but with the benefit of not having to actually modify the feed itself.

    One could also write their own product-specific supplimental content for each product the same way.....
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  8. #8
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    The only proper (or should I say "effective") way to do it, would be to compare the new feed with the old one. Ideally, one could then select the records to modify/update/tweak. WM would then only generate pages based on what gets modified.

    I would hate (and do hate) having to overwrite pages that are arranged/tweaked just right - especially ones that are generating organic search traffic.

    That's your challenge Richard.

    Fred

  9. #9
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    The other tool that we have talked about before, is a very powerful search and replace function. I'm not talking about some wimpy "swap out slashes for dashes" tool. The tool needs to be able to take 3k or 4k worth of text/per record and add/subtract everything from HTML tags to URLs based on special trigger words/phrases. Currently, (in Excel 2k) Excel's S&R function chokes when you have more than about 700bytes of info in a field.

    WM needs to have an interface similar to excel, so that the entire record can be viewed and individual fields modified. Right now, I can't see the entire record - no horizontal scroll bar.

    Okay, how's that for comments outside the scope of the thread?! Sorry - pet peeve.

    Fred

  10. #10
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    > Okay, how's that for comments outside the scope of the thread?! Sorry - pet peeve.

    Not at all, but that may wind up in v3.0 rather than v2.5. Definitely on the plate, though.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  11. #11
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    WM needs to have an interface similar to excel, so that the entire record can be viewed and individual fields modified.
    I'm willing to pay some extra for that feature, but only if I am going to be able to resize the window of the WM too.

    Sal.

  12. #12
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Even better: it'll be in a resizable window, but a separate one from the main WebMerge window so you'll have all the room for viewing your data without clutter from the other WebMerge controls.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  13. #13
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795

    Quote Originally Posted by FourthWorld
    Even better: it'll be in a resizable window, but a separate one from the main WebMerge window so you'll have all the room for viewing your data without clutter from the other WebMerge controls.
    That would be great, thanks.

    Sal.

  14. #14
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    And... Resize the WM window as well. Please.

    Fred

  15. #15
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Quote Originally Posted by buy_online
    And... Resize the WM window as well. Please.
    I don't understand. There's only one field in WebMerge which could benefit from resizing the window, and I'm giving you a solution which gives you a better view with more space -- why would the WebMerge window still need to be resized?
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  16. #16
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    why would the WebMerge window still need to be resized?
    One reason may be because not everyone have the same screen size monitor.

    My reason still the same one I requested about a year ago ...

    I like to be able to resize the windows of some programs that don't open all the way from the top and leave a margin gap on the top and left side of the screen.

    Sal.

  17. #17
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Quote Originally Posted by Mr. Sal
    One reason may be because not everyone have the same screen size monitor.
    Of course. WebMerge's controls fit comfortably within a window that will work well on any monitor from 800x600 on up.

    Quote Originally Posted by Mr. Sal
    My reason still the same one I requested about a year ago ...

    I like to be able to resize the windows of some programs that don't open all the way from the top and leave a margin gap on the top and left side of the screen.
    Even after I deliver a data viewing window whose layout is optimized for that task, WebMerge will not be able to affect the resizing of windows in any other programs.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  18. #18
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    Richard,

    MS Windows viewing habit are as personal as could be, everyone is different - that's the easiest answer I could give. There is an entire psychology behind the GUI.

    From what I remember, it is not that difficult to allow the user to change the size of the window - why not let them do it? They aren't going to hose something up. Let's face it, no one is going to have their resolution set so small that WM won't fit - but people still like to change the shape and size of the window, depending on what they are doing. For example, if I am working on a database, and tweaking different records, I like to maximize the application - I don't need all the other distractions on my desktop.

    Just because you have designed your app to work with your own work habits does not mean that they are the same work habits as anyone else. This is the part of the design job that isn't coding, it's subjective, right-brain stuff - so everyone is different.

    Fred

  19. #19
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Quote Originally Posted by buy_online
    Richard,

    MS Windows viewing habit are as personal as could be, everyone is different - that's the easiest answer I could give. There is an entire psychology behind the GUI.
    As a student and practitioner of interface design for 17 years, I agree wholeheartedly.

    But in order to design a solution for a request I need to understand how it will be used.

    If I make the window resizable, what do I do with the controls in the window? Do I stretch them, leave them in place with an empty gap at the right and bottom, or center them with an equal gap on all sides?

    If the benefit you're looking for is to obscure windows from other apps behind WebMerge, I can make a plugin to put up a backdrop in about 5 minutes.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  20. #20
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    The answer is this.

    Leave everything at the top-left of the window. Also, in v2.4 you would allow the window for previewing the data feed file to expand. And add a horizontal scoll to that window.

    Fred

  21. #21
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Quote Originally Posted by buy_online
    The answer is this.
    Leave everything at the top-left of the window.
    Done:
    http://www.fourthworld.com/products/...ndowResize.4wt

    Put that in your Tools folder and when you launch WebMerge you can choose it from the Tools menu. The change is not currently persistent between sessions (the next version of the plugin architecture will provide the deeper hooks to make that possible), but at least for now you can have a WebMerge window with lots of unused empty space to the right and bottom as requested.

    Quote Originally Posted by buy_online
    Also, in v2.4 you would allow the window for previewing the data feed file to expand. And add a horizontal scoll to that window.
    v2.4 is already shipping.

    When I deliver a version with the Database Viewer window there will be no benefit to showing multiple records in the delimiter preview, but there would be a detriment to doing so:

    The preview is shown there to let you know if your delimiters are set correctly. That's why it only shows the field names and the first data record. It works on only a small portion of the file in order to display your changes quickly.

    If I chose to alter that to have it also serve the very different purpose of acting as a general database viewer, it would need to process the entire file for display each time any control in that tab is changed. So what is now an almost instantaneous change to let you know what your delimiter settings are doing with the data would become a wait of several seconds (possibly much longer depending on the feed size and its format).

    You gotta trust me on this Fred: I want to completely address the task-based reason behind your request, but giving you a compromised solution which also slows down everyone else isn't optimal. It's better to give you a separate window designed specifically for that task, leaving the delimiter preview area quick and responsive. Two very different tasks, two different interfaces for doing them.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  22. #22
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    Richard, I just tried the new WindowResize.4wt tool and I like it.

    Thanks.

    Sal.

  23. #23
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Quote Originally Posted by Mr. Sal
    Richard, I just tried the new WindowResize.4wt tool and I like it.

    Thanks.
    You're welcome. Only took a few minutes to make.

    I could make a backdrop option also, so that rather than resizing the window it would put up a backdrop to cover any other programs running behind it.

    But I'll be the first to admit I don't grasp the benefit here, so forgive me if the suggestions seems off base.

    Most control panels, Norton, and thousands of other apps used fixed-sized windows, and since WebMerge's controls don't benefit from resizing I've been at a loss to understand the subjective attraction to doing so.

    Could I trouble you to tell me what you like about it? It'll help me make a better one.

    Thanks in advance -
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  24. #24
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    tell me what you like about it?
    While that feature may not be to important to many users, I like to have the entire WM window open to cover the entire screen and not have to see whatever it's behind the WM window at that time, show around all four sides.

    Now with the new tool, when I open WebMerge, I only see the WebMerge Window, so I can concentrate more on the datafeeds and not on what's behind.

    Also, now I don't have to move the WM window UP a little in order to see and click on the build pages button.

    Sal.

  25. #25
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Quote Originally Posted by Mr. Sal
    While that feature may not be to important to many users, I like to have the entire WM window open to cover the entire screen and not have to see whatever it's behind the WM window at that time, show around all four sides.

    Now with the new tool, when I open WebMerge, I only see the WebMerge Window, so I can concentrate more on the datafeeds and not on what's behind.
    Sounds like the backdrop option will be useful. I'll see if I can do that in the next day or so.

    Quote Originally Posted by Mr. Sal
    now I don't have to move the WM window UP a little[/b] in order to see and click on the build pages button.
    That should never be needed on any display 800x600 or larger.

    What's your display's resolution? If it's within the supported range I'll happily tidy up the window placement to address that.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

+ Reply to Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. How much percentage goes into marketing?
    By Xion in forum Building Traffic, Newsletters & Advertising
    Replies: 7
    Last Post: October 8th, 2006, 10:16 PM
  2. Adsense - What percentage do we get?
    By oranges in forum Other Affiliate Networks
    Replies: 15
    Last Post: September 25th, 2003, 03:30 PM
  3. What percentage with script
    By philr in forum Cusimano.com Scripts
    Replies: 1
    Last Post: September 8th, 2003, 08:11 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •