View Poll Results: Poll - How do you get your data feed?

Voters
44. You may not vote on this poll
  • Download via FTP

    23 52.27%
  • Download via the Web

    13 29.55%
  • Sent to me as an email attachment

    2 4.55%
  • Exported from my own database/spreadsheet locally

    5 11.36%
  • Other (please explain)

    1 2.27%
Page 1 of 2 12 LastLast
Results 1 to 25 of 36
  1. #1
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Please select the option below which best describes how you obtain the data feeds you use with WebMerge:
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  2. #2
    Full Member
    Join Date
    January 18th, 2005
    Posts
    291
    can you add *multiple ways*? Or am I misunderstanding the question?

    Bfree I get ftp
    most of cj I get either as an email attachment or d/l it right from their site.

    All of them are imported to excel, cleaned, then used from that saved file with webmerge.

  3. #3
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    I voted for via FTP, because that is how I get most of my feeds, but the actual answer is all 4 ways you have listed. I have indy feeds that are downloaded via the web, one feed that is emailed to me weekly, and am beginning to create my own with Excel.

    ALL feeds have to go through post processing in Excel before they can be used, mostly to add id, concatenate fields for categories and descriptions and strip out illegal characters.

    For WebMerge I will be using the non-network feeds, so it will be the ones that are downloaded via the web or the ones I create in Excel. I guess I should have voted the other way!!! LOL. I've been house hunting today and my brain is lagging behind.... It's on floor plan overload....
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  4. #4
    ABW Ambassador Ron Bechdolt's Avatar
    Join Date
    January 18th, 2005
    Location
    Affiliateville, USA
    Posts
    7,927
    You need an option that says All of the Above. I for one use A, B and C options.
    Ron Bechdolt | Affiliate Program Management Consultant
    7 Days A Week Marketing

  5. #5
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    In order of useage:

    1. FTP from Merchant Server
    2. Merchant server to my server, and FTP to me.
    3. E-Mail

    Just the way it falls here...

    Fred

  6. #6
    Member MarkJH's Avatar
    Join Date
    January 18th, 2005
    Posts
    68
    Wow. Am I the only person who actually works with their own data?
    [b][url=www.bandlink.net]Bandlink.net[/url][/b]

  7. #7
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by MarkJH:
    Wow. Am I the only person who actually works with their _own_ data? <HR></BLOCKQUOTE>

    Did you miss that part of my post? I am starting to work with MY OWN data.

    Most people affiliated with this forum use WebMerge to process merchant data feeds, so do NOT need to use it for "their own data". That is the nature of ABW.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  8. #8
    Super Sh!t Stirrer SSanf's Avatar
    Join Date
    January 18th, 2005
    Posts
    9,944
    All of the Above
    Comments are opinion unless otherwise noted. Remember, pillage first. Then burn. Half of all people in the world have IQs under 100. You best learn to trust ol' SSanf!

  9. #9
    Affiliate/AM Moonlighter dflsports's Avatar
    Join Date
    January 17th, 2005
    Posts
    874
    Another vote for all of the above.

  10. #10
    Member MarkJH's Avatar
    Join Date
    January 18th, 2005
    Posts
    68
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Did you miss that part of my post? I am starting to work with MY OWN data <HR></BLOCKQUOTE>Guess I missed that bit.
    [b][url=www.bandlink.net]Bandlink.net[/url][/b]

  11. #11
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    Hmmm....my aim for this poll was to see if I should add support for automatically downloading feeds to process, eliminating one manual step.

    But if most of you are also modifying the feed in Excel or other tool, then there would be little point in adding that until I add the ability to automate feed modifications.

    Is my understanding correct, that most of you modify your feeds before processing?
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  12. #12
    Full Member
    Join Date
    January 18th, 2005
    Posts
    291
    I've never gotten one yet that was perfect as far as extra characters. Others doing it longer than me might differ.

  13. #13
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Is my understanding correct, that most of you modify your feeds before processing? <HR></BLOCKQUOTE>

    A large portion of the feeds need the affiliate code added at the very least. ALL need to be sorted to properly produce categories. So in a word YES.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  14. #14
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    I modify every single feed. Some for SEO, some because the merchant has provided a piece of ***t right out of the box, and almost all in order to make mine unique once published.

    Fred

  15. #15
    ABW Ambassador Mike O's Avatar
    Join Date
    January 18th, 2005
    Location
    Los Angeles area
    Posts
    843
    The feeds *always* need cleaning, or sorting, or something...

    -- Mike
    A joy shared is a joy doubled.
    A burden shared is a burden lightened.

  16. #16
    Full Member
    Join Date
    January 18th, 2005
    Posts
    413
    I access almost all of my feeds via FTP (BeFree and CJ), although I do use "all of the above" on occasions.

    The only feeds that I use "as is" (without cleanig, inserting and sorting) are the ones that run online using a web generation script provided by a third party.

  17. #17
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by Ms. Woodtick:
    I've never gotten one yet that was perfect as far as extra characters. Others doing it longer than me might differ. <HR></BLOCKQUOTE>

    What sort extra characters are in the feeds you work with?

    My goal is to provide a turnkey solution for automating such changes, so I'm very keen on providing an interface for all of the modifications that need to be done.

    Thanks in advance for helping me get a good understanding of what's needed.
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  18. #18
    Full Member
    Join Date
    January 18th, 2005
    Posts
    291
    /, &, &#169;, all the numeric codes etc. I tried leaving numeric codes and it printed them as such. I'm sure if I was more experienced I could get around that, but I'm still a babe in the woods with most of this stuff.

  19. #19
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    "What sort extra characters are in the feeds you work with?"

    Depends!

    For example, if I am using category field names in order to generate a filename, and there are slashes in the records for that fieldname, then the results are disastrous - WM starts creating directories.

    If I have an & # 153; (spaces included to show in the post) in the description, that might work ok when it gets printed out on the user's page, but in a meta tag, I would rather not have it. I would rather have " ™ " instead.

    The list goes on and on. That's one reason why a search and replace function is very important.

    Yes, I can use the "replace" function, but it's one more step, and gets very cumbersome, when ganging several together.

    Fred

  20. #20
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by MikeO:
    The feeds *always* need cleaning, or sorting, or something...

    -- Mike <HR></BLOCKQUOTE>

    What type of sorting capabilities would you need to do that in WebMerge?
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  21. #21
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by buy_online:
    _"What sort extra characters are in the feeds you work with?"_

    Depends!

    For example, if I am using category field names in order to generate a filename, and there are slashes in the records for that fieldname, then the results are disastrous - WM starts creating directories.

    If I have an & # 153; (spaces included to show in the post) in the description, that might work ok when it gets printed out on the user's page, but in a meta tag, I would rather not have it. I would rather have " ™ " instead.

    The list goes on and on. That's one reason why a search and replace function is very important. <HR></BLOCKQUOTE>

    Should it be global, or would it be important to allow it to be restricted to a given column?

    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Yes, I can use the "replace" function, but it's one more step, and gets very cumbersome, when ganging several together. <HR></BLOCKQUOTE>

    What steps are required how you're doing it now, and in what program?
    Richard Gaskin
    Developer of WebMerge: Publish any data feed on any site
    http://www.fourthworld.com

  22. #22
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    Most of my feeds need changing. First is the PID. If I'm putting it anywhere besides my default site, that needs changing. Then the slashes need to be dumped out of whatever field I use for the page names.

    There are more hokey characters than those previously mentioned. They appear to be foreign characters, but I don't know the exact names of them. It's not something you'd see in English, though, that's for sure!

    These foreign characters are NOT supposed to be in there, but they often are. Especially right before a copyright seal.

    Other problems common to feeds include fields being in ALL CAPS, and Program Names that are anticonversionary ("So and So's Affiliate Sales Program). Both these things get changed.

    ~~~~~~~~~~~

    For the smaller feeds I change them in Works Spreadsheet, but for changes that effect my entire CJ Feed I end up using MySQL. It can F/R my PID across 1 GB+ of data in about 10-20 seconds.

    Unfortunately, I often forget to take out the hokey characters that I don't happen to visually spot when I'm running it.

    Also, it's a seperate operation to remove each character type. For instance, first I get rid of (Spaces included to make the code show) "& q uot;" then remove "& a pos;" and on and on. That's a tiring pain in the rear!

    And if I use mySQL to do it, of course I usually forget all about those kinds of characters (since I don't see them like I would in a spreadsheet) until I've put the pages live and see them looking back at me.
    There is no knowledge that is not power. ~Hemingway

  23. #23
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    "Should it be global, or would it be important to allow it to be restricted to a given column?"

    Restricted to columns (field names) would be great!

    "What steps are required how you're doing it now, and in what program? "

    That's a little bit of a secret, but I was thinking of publishing at my site anyway. An example would be generating a set of keywords, or if you will, a new field name [column] to be used to place keywords in the meta tag. (Done with Excel).

    For example, all of the words like "the" and "a" and many many others need to be stripped out of a description. All of the words need to be changed to lower case. Some of the spaces need to be removed, and commas need to be put in their place. I say some, because isn't it great to have a few phrases in your keywords (rhetorical question)? While we're at it, let's convert quotation marks to the word "inches." Adding HTML formatting to descriptions.

    There are more, but you get the idea. If merchants knew that we were not updating their feeds every two days in order to get this type of quality into a page, I wonder what they would say? Look at Back C*untry St*re, they get all excited if you don't update every week - not going to happen. I think Leader will agree about removing prices too.

    Hope I haven't confused matters further.

    Fred

  24. #24
    ABW Ambassador Mike O's Avatar
    Join Date
    January 18th, 2005
    Location
    Los Angeles area
    Posts
    843
    Usually sorting by fields (column contents), 3-4 levels deep maximum. I've never sorted on more than three fields so far, but you never know.

    But sorting is easy in Excel or whatever. What's more important to me is to be able to repeat the "cleaning" steps in an automated fashion. For me that's mainly replacing improper characters or character strings with proper ones, in only the specified fields or globally, at my choice.

    It would even be nice to be able to use a separate file of search/replace commands, or even string multiple such files together.

    Being able to use an external search/replace command file would let us develop a general s/r command set and apply it to various vendors' feeds.

    I'd even be happy to just be able to "import" and "export" the S/R steps I'm using from/to a file. That way they could stay "internal" to WM, yet be transferrable for use with new datafeeds from different vendors as they became available.

    It would also be very useful to be able to turn off the s/r feature, for example in the case that I'd just made a small update to a datafeed and didn't need/want to run the s/r operations again.

    Quite a wish list, I know!

    -- Mike
    A joy shared is a joy doubled.
    A burden shared is a burden lightened.

  25. #25
    ABW Ambassador buy_online's Avatar
    Join Date
    January 18th, 2005
    Location
    Richmond, VA
    Posts
    3,234
    a little more...

    One of the limitations in excel, is delimiters when sorting columns for the operations "text to columns" and when combining columns. I would love to be able to use a phrase as the delimiter in a text-to-column operation for example. Currently Excel only allows a single character, and that of course cannot be a wildcard.

    So, in addition to search and replace, text-to-columns would also be important. Many data feeds use a single column for all of the categories, separated by delimiter (like "&gt;&gt;"). These need to be separated into columns (and named) prior to running WM.

    Fred

+ Reply to Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Extract Data from a Data Feed
    By TalhaA in forum Programming / Datafeeds / Tools
    Replies: 6
    Last Post: February 25th, 2012, 05:40 PM
  2. Data Feed - Invalid Compressed Data in .gz files?
    By Bleddyn in forum Rakuten LinkShare - LS
    Replies: 7
    Last Post: November 29th, 2010, 09:30 AM
  3. data feed not there?
    By vertygo in forum Commission Junction - CJ
    Replies: 18
    Last Post: September 18th, 2004, 05:57 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •