Page 1 of 2 12 LastLast
Results 1 to 25 of 32
  1. #1
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Integrating AvantLink Datafeeds?
    I've been working on trying to integrate automated downloading and import of datafeed updates, and unfortunately I am continuing to have trouble figuring out how I might do this with AvantLink.

    Among the issues I'm confronting:
    • The need to manually configure each datafeed for each merchant;
    • The lack of any merchant-id or merchant-name field within the datafeeds;
    • Inconsistency between datafeeds -- different fields in different sequences;
    • The existence of multiple overlapping (partially duplicate) datafeeds for some merchants.

    This has been a very interesting experience; when I first started working with a few AvantLink datafeeds for specific merchants (creating a new WebMerge template for each merchant -- was that really two years ago?), everything seemed fine -- but I'm no longer using WebMerge and I am not doing any work that does not allow for fully-automated updating.

    I can't seem to discern any workable strategy to "scale" from 1, 2, or 3 AvantLink datafeeds to several dozen (or 120+). In contrast, my system is now importing datafeeds from 436 ShareASale merchants.

    My initial thought was to just use AvantLink merchants via PopShops, but when I just checked, I see that AvantLink datafeeds aren't available through PopShops, either.

    Fortunately, many AvantLink merchants also offer affiliate programs through other networks, so hopefully I will still be able to include most of these merchants in my projects.

    If someone has some specific "tips or tricks" that might allow me to manage and import AvantLink datafeeds, I'd love to hear your advice. For now, I'm dropping AvantLink from my future efforts.
    Last edited by markwelch; April 21st, 2009 at 06:51 PM.

  2. #2
    Influencer Marketing GravityFed's Avatar
    Join Date
    January 18th, 2005
    Location
    Ithaca & Park City
    Posts
    3,338
    Mark, David did get your follow-up emails today, but he won't be able to respond until the morning... GM

  3. #3
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    England
    Posts
    4,327
    Have you made any money from any of those 436 merchant datafeeds?

    I really like working with the Avantlink datafeeds. Dropping them completely seems to be rather extreme.

    The way I work with the feeds may not be anything like you are looking for, but I can now update over 100 feeds, but then I am using Excel, Dreamweaver and Webmerge

  4. #4
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    Mark I'm a datafeed dummy and only managed my couple with a lotta help from BurgerBoy. (big note* Am sure if I had asked AvantLink for help they would've but I was pigheaded and wanted to at least try to do it on my own first)

    That said, I wouldn't have a clue to help you. Maybe BurgerBoy will see this tho and have an idea cos he's a real datafeed smarty but if not......

    If there is a possible way to do what you wanna do I'm sure if you ask Gary or Scott or anybody really at AvantLink cos am sure they'd get you the person to talk to, they will help you. Honest they bend over backwards to try to help. I've had some questions on things not datafeed related and have been amazed how quickly they got back to me and helped. (wouldn't be in your case cos you're also one of the computer smarties but in my case, when they've helped me they don't make me feel like a dummy for having asked them, either)

    Good luck

    Lol you guys answered before I finished mine. lol Told ya they'd get back to ya fast. lol

  5. #5
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Joshua: Yes, I've earned advertising revenue from a number of those datafeeds, but certainly not a significant amount (recently, just a few transactions per week, most driven through PopShops pages). As I've mentioned often on ABW, I'm not nearly finished implementing my datafeed-management system, and over the past 18 months I've done almost nothing to promote any of my sites, so my advertising income has shriveled.

    I'm not "dropping" AvantLink datafeeds -- there's nothing to drop. The issue is whether there is any efficient way to add them into my system and facilitate automatic updating.

    Once I've dealt with automating feeds from other networks where I can handle dozens or hundreds of merchants with a single script, then I'll probably re-visit some AvantLink-only merchants -- but that certainly won't be in 2009.

    Or perhaps someone will be able to explain to me that I'm mistaken about all these obstacles that I perceive.

  6. #6
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    Quote Originally Posted by markwelch
    I've been working on trying to integrate automated downloading and import of datafeed updates, and unfortunately I am continuing to have trouble figuring out how I might do this with AvantLink.

    Among the issues I'm confronting:
    • The need to manually configure each datafeed for each merchant;
    • The lack of any merchant-id or merchant-name field within the datafeeds;
    • Inconsistency between datafeeds -- different fields in different sequences;
    • The existence of multiple overlapping (partially duplicate) datafeeds for some merchants.
    This has been a very interesting experience; when I first started working with a few AvantLink datafeeds for specific merchants (creating a new WebMerge template for each merchant -- was that really two years ago?), everything seemed fine -- but I'm no longer using WebMerge and I am not doing any work that does not allow for fully-automated updating.

    I can't seem to discern any workable strategy to "scale" from 1, 2, or 3 AvantLink datafeeds to several dozen (or 120+). In contrast, my system is now importing datafeeds from 436 ShareASale merchants.
    Mark, if you can do it with SAS datafeeds, you can do the same with AvantLink merchants datafeed.

    I used to have some os those same issues with some of the SAS datafeeds, but I found a way around those same issues.

    When I find a merchant with different inconsistencies between the other datafeeds, if I really want to work with that merchant I just adapt my field names to that merchant on one separate script just once.

    My initial thought was to just use AvantLink merchants via PopShops, but when I just checked, I see that AvantLink datafeeds aren't available through PopShops, either.

    Fortunately, many AvantLink merchants also offer affiliate programs through other networks, so hopefully I will still be able to include most of these merchants in my projects.
    Well, since i have learned to make and modify my own scripts, I no longer worry if the merchants have their datafeed thru another party or not...

    But here is one tip: If I am having problems creating a datafeed from one of the merchants that I want to work with, I do check to see how the datafeed is used on PopShops or GoldenCan, and if I see that they can display it, then I work my way into fixing my scripts because I see it can be done...

    If someone has some specific "tips or tricks" that might allow me to manage and import AvantLink datafeeds, I'd love to hear your advice. For now, I'm dropping AvantLink from my future efforts.
    Mark, by now I know that you want to have a system on were you can have almost every merchants datafeed from many networks, and I also know that you have wasted a lot of money trying those other so called solutions that you have decided they don't do exactly what you want to do...

    So here is one more "tip or trick", for you...

    If by now you're able to handle the SAS datafeeds by automatically downloading and importing the datafeed updates, then you can easily do the same with many of the AvantLink datafeeds...

    I'm not that good at PHP and Mysql yet , but if I can do it, I think you can do it too...

    Look, I am not going to talk about all the time I wasted the first time I tried to use my first AvantLink datafeed , not because it was it was so difficult to use, but because at that time I was not too sure about the difference between: datafeeds.shareasale.com and avantlink.com/download_feed.php

    One final tip:
    If someone has some specific "tips or tricks" that might allow me to manage and import AvantLink datafeeds, I'd love to hear your advice.
    With a little modification, you can use the same script for SAS datafeeds, that is posted here on ABW, all you need to do is replace the Field names from the SAS format to the avantlink format, and of course you need to change the way to download the feed too. But you can use the SAS template script as a road map to get there...

    As for the "automated downloading and import of datafeed updates",that have nothing to do with what network you get the feed from, that's the cronjob, Job.

  7. #7
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    > "If by now you're able to handle the SAS datafeeds by automatically downloading and importing the datafeed updates, then you can easily do the same with many of the AvantLink datafeeds... if I can do it, I think you can do it too..." <

    The key difference is that with SAS, all feeds have the same data fields, in the same sequence. With AvantLink, that's simply not true. I know I could create an import script for one merchant, but the time required to manually configure every single merchant, and then manually modify the script over and over, is simply not justified.

  8. #8
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    The key difference is that with SAS, all feeds have the same data fields, in the same sequence
    That's simply not true!

    I will give you just one example: KiteAndWind uses Custom4 and Custom5 as category and subcategory as field names, when the mayority of the other feeds use Custom1 and Custom2 as category and subcategory as field names...
    I know I could create an import script for one merchant, but the time required to manually configure every single merchant, and then manually modify the script over and over, is simply not justified.
    I don't know why you would need to manually modify the script over and over for any merchant, because I surely would not do that for any merchant anywhere.

    Maybe it's the way you use the same script for every merchant thinking that the're all created equal, but by now I know better than that...

    For example, on SAS I know that many merchants have adapted the new datafeed format, so I know that all I have to do is just change the merchant ID# on my master script, but once in a while I find some merchants datafeed like the KiteAndWind and I have to decide if I want to add that merchant or not, but since I will only have to do it once, I just add something like this to my script:
    if ($data[2] == "xxxxx")
    {
    $category = $data[15];
    $sub_category = $data[16];
    }
    I think the big difference is that I only join a merchant after I check them out or read about them first, but since I think you just like to add every merchant available on the network just because they have a datafeed, that maybe you're right and it's just simply not justified for you to take the time to check their datafeeds first.

  9. #9
    OPM and Moderator Chuck Hamrick's Avatar
    Join Date
    April 5th, 2005
    Location
    Park City Utah
    Posts
    16,646
    Mark, I find this rather amazing since the AvantLink standard is based on a two year discussion here on ABW by datafeeders. Are you trying to fit a square peg into a round hole? I will wait to hear what David from AvantLink or Scott has to say.

  10. #10
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Mr. Sal, I appreciate your comments, but I simply don't think it is appropriate for me to use a discussion thread in the AvantLink forum to discuss the issues involved with other networks' datafeeds, nor to expound on my overall philosophy nor my specific projects.

    The issue of how each merchant fills "custom" fields that always appear in the same columns in a datafeed is a very different issue than, for example, discovering that the field for suggested-list-price is column 8 in one datafeed, but it's in column 7 in another datafeed, and it's not present at all in another datafeed.

    Chuck, maybe I am trying to "fit a square peg into a round hole," since I really cannot wrap my mind around the AvantLink datafeed "model." It appears to be "very powerful and very flexible," but for me, the flexibility is proving to be a complete barrier, rather than an advantage.
    Last edited by markwelch; April 22nd, 2009 at 01:15 AM.

  11. #11
    Affiliate Network Rep
    Join Date
    August 20th, 2007
    Posts
    6
    Mark,

    I'll do my best to cover the points you've raised below.

    The lack of any merchant-id or merchant-name field within the datafeeds:
    We do not include the merchant id or merchant name as a field within a datafeed because it would be the same repeated value throughout the full feed. This is redundant and not in keeping with the per-product nature of our feeds. However, if you have a list of datafeeds that need to be retrieved and imported it should be possible to include information about what merchant that feed is for in your import mechanism. We also provide a "SubscriptionFeed" API module that helps simplify this process - matching up datafeeds to merchants, and showing various supplemental information.

    Inconsistency between datafeeds -- different fields in different sequences:
    We encourage all merchants to supply as complete a datafeed as possible. Realistically however, some merchants are simply unable to supply as much data as others, e.g. some may not have brand name
    information, others may have only 1 level of product categorization instead of 3. Some merchants may have much more information available (UPCs, color swatches, product reviews, etc.) than the norm. Rather than force everyone to a least-common-denominator set of information (where all merchants would supply only the bare minimum number of fields), we allow merchants to supply a standard/required set
    of fields, and then any number of optional/extended attributes that they feel may be useful to some affiliates.

    This means that yes, there are differences among datafeeds between different merchants in AvantLink.com. But for mass consumption of those datafeeds this is not necessarily an insurmountable obstacle.

    For instance, during the configuration of your feed, if you choose the "All datafeed columns" option (which is the default) then the first 19 fields in the datafeeds will always be exactly the same. All optional/custom fields from merchants are appended thereafter. So you could modify your import routine to ignore any fields after the 19th.

    Alternatively, it is true that you if you choose specific fields/column to include in a datafeed that you cannot control the order of those fields (though for a given datafeed, the order will always remain constant). Still there are several options for handling this, including:
    1. Adjust your import software to use field names instead of field positions. All AvantLink.com datafeeds always use the exact same field names for all standard fields.
    2. Configure your AvantLink.com datafeeds in XML format, and then run a minor XSLT on your end to transform the XML into a delimited format that works with your import mechanism.


    The existence of multiple overlapping (partially duplicate) datafeeds for some merchants:
    We allow merchants to provide multiple datafeeds, in particular "specialty" datafeeds that highlight specific sets of products (e.g. "Eco-friendly products", "Best sellers", etc.). Since you choose which datafeeds to pull from AvantLink.com you can choose to ignore these types of sub-feeds.

    The need to manually configure each datafeed for each merchant:
    To clarify this point - we only support the ability to build/configure datafeed subscriptions one-at-a-time (not in bulk). So if you want to download 100 different datafeeds from AvantLink.com you will first have to configure 100 different subscriptions.

    For an affiliate who builds highly targeted sites this is likely not much of an issue - s/he would want to put some time into choosing which datafeeds should be presented on their site, and potentially add in filters to further tighten the scope of the presentation.

    From the standpoint of someone who is consuming a mass number of datafeeds without any filtering (say, all products from all available datafeeds) this is obviously a downside. Again, not insurmountable (a few minutes per datafeed), but bothersome. However, such a position does raise the question: what is being done with all of these datafeeds? If they're being used in a search-engine-type catalog site, perhaps our "ProductSearch" API module would be more efficient (no subscriptions required, products from multiple merchants can be returned from a single request, etc.). Similar non-subscription options exist for RSS product feeds as well, which may be a good alternative.

  12. #12
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    David suggested: > "Adjust your import software to use field names instead of field positions" <

    Quite frankly, I tried this, and it is simply something that I just can't seem to do correctly. (PHP documentation is extraordinarily terse; the vast collection of resources about PHP are horrifically scattered, inconsistent, riddled with errors, and ultimately prove to be a waste of time more often than not. In this case, I spent many hours trying several different strategies which were clearly contradictory, and none of them worked for me, and eventually I concluded that it was extremely unlikely that I'd ever figure this out unless I could find and hire a competent adviser, which I've clearly proven I am incapable of doing.)

    David wrote: > "if you choose the "All datafeed columns" option (which is the default) then the first 19 fields in the datafeeds will always be exactly the same" <

    I didn't know this. If I had known, I would have saved a lot of time (a couple of dozen hours of failed attempts to "use field names instead of field positions," and some number of hours manually trying to configure many datafeeds).

    I know that I've been increasingly frustrated and scattered as I've tried to move this project forward (I've probably lost a lot of brain cells after countless hours of banging my head against walls and desks in frustration), but I honestly don't think I would have missed this important tidbit. Was this documented somewhere on the AvantLink web site, or in your prior email responses to my questions? (If it was, then maybe it's time for me to simply accept that my brain is no longer working properly, and abandon this entire project.)

    The manual configuration thing is definitely still a huge problem; it really does take 3-4 minutes to configure each datafeed, and then hours to figure out which datafeeds I configured incorrectly. It's very cumbersome, and I think very needlessly so.

    > "From the standpoint of someone who is consuming a mass number of datafeeds without any filtering (say, all products from all available datafeeds) this is obviously a downside. Again, not insurmountable (a few minutes per datafeed), but bothersome. However, such a position does raise the question: what is being done with all of these datafeeds?" <

    I'm getting sick and tired of that question. I've disclosed an awful lot about my efforts and plans over the past 18 months; I'm not wasting any more time trying to "justify" getting access to data.

    On the merchantname/merchant-id issue, I cannot discern any way to create a useful list of merchants, since there is no consistent merchant-id tag. I failed to find the API that would hand me a list of merchants and the associated datafeed subscriptions, but again it seems like maybe I'm just losing my mind.

  13. #13
    OPM and Moderator Chuck Hamrick's Avatar
    Join Date
    April 5th, 2005
    Location
    Park City Utah
    Posts
    16,646
    I'm getting sick and tired of that question. I've disclosed an awful lot about my efforts and plans over the past 18 months; I'm not wasting any more time trying to "justify" getting access to data.
    Seems difficult to offer support if you are being evasive! You post on this forum rather than email directly or pick up a phone. That said, this is AvantLink's forum so I will shut-up and

  14. #14
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Evasive? I've posted a lengthy discussion thread and series of very detailed specification documents, explaining the exact strategy in more detail than I've ever seen any other web publisher do. I certainly shouldn't need to do so -- but I did so.

    > "You post on this forum rather than email directly or pick up a phone." <

    I've tried to get help directly from AvantLink a number of times, using both their trouble-ticket system (which they don't seem to actually use) and emails. While I've received some responses, I have not been able to figure out what I need to do to get practical access to import their datafeeds. And please let me know if YOU can find a phone number anywhere on the AvantLink web site.

  15. #15
    OPM and Moderator Chuck Hamrick's Avatar
    Join Date
    April 5th, 2005
    Location
    Park City Utah
    Posts
    16,646
    Fair enough, they have communication restricted to a trouble ticket and even though they are active here that is the only contact via their site.

  16. #16
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    My mistake: datafeeds can have fixed column sequence
    I've reviewed my past communications with AvantLink, and sure enough, David did clearly inform me (last August) that if I selected "all columns" then the first 19 columns in the datafeed would always represent the same exact sequence (although some would be blank for some merchants).

    "Standard" fields: SKU, Manufacturer Id, Brand Name, Product Name, Long Description, Short Description, Category, SubCategory, Product Group, Thumb URL, Image URL, Buy Link. Keywords, Reviews, Retail Price, Sale Price, Brand Page Link, Brand Logo Image, Product Page View Tracking
    David also included a phone number in emails last year, but not in more recent emails.

    I really do think that there is something wrong with my brain -- I can no longer recall stuff like this, nor find it when I try to do so (and I really did read through many prior AvantLink emails and tried to research afresh last night before posting this thread).

  17. #17
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    ok, everybody count to 10, please

    Mark, I know you're probably very frustrated cos you're trying to do something and can't seem to be able to quite figure that out but just lookin at this as purely an outsider......umm I think you're taking David's words just a teeeeeeny bit too sensitively.

    David went on to say after that:
    perhaps our "ProductSearch" API module would be more efficient (no subscriptions required, products from multiple merchants can be returned from a single request, etc.). Similar non-subscription options exist for RSS product feeds as well, which may be a good alternative.

    As to the trouble tickets. Not sure why there seems to be a problem with yours but I've used it several times and I always get very prompt answers to my questions.

    If you want to do your datafeeds with AvantLink, please just for a minute everybody take a deep breath and just start to try to figure out a solution to your problem now.

    Honest Mark I have absolutely no problems at all getting help from AvantLink. I'm sure they will try to help you with your problem

    Am not trying to butt in or anything, just as an outsider thought you might like to hear my experience with them. They really will try to help you if there is a way to do it. Good luck with it

  18. #18
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    England
    Posts
    4,327
    "I really do think that there is something wrong with my brain -- I can no longer recall stuff like this, nor find it when I try to do so"

    datafeed overload, working on too many datafeeds, and trying to deal with too many issues related to datafeeds. It fries your brain

  19. #19
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    Mark
    psssssst it's called gettin older.....we all have them moments where we forget things Good luck fixing your problem

  20. #20
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    Julian
    I hardly do datafeeds at all and my brain's fried most of the time what's your excuse for me

  21. #21
    Moderator BurgerBoy's Avatar
    Join Date
    January 18th, 2005
    Location
    jacked by sylon www.sylonddos.weebly.com
    Posts
    9,618
    Mark

    Seems to me like you have an advanced case of CRS (Can't Remember Schitt).

    Vietnam Veteran 1966-1970 USASA
    ABW Forum Rules - Advertise At ABW

  22. #22
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    England
    Posts
    4,327
    Quote Originally Posted by purplebear
    Julian
    I hardly do datafeeds at all and my brain's fried most of the time what's your excuse for me
    teddy bear overload

  23. #23
    Moderator BurgerBoy's Avatar
    Join Date
    January 18th, 2005
    Location
    jacked by sylon www.sylonddos.weebly.com
    Posts
    9,618
    Talking
    Quote Originally Posted by Julian
    teddy bear overload
    That - and being a Yankee.

    Vietnam Veteran 1966-1970 USASA
    ABW Forum Rules - Advertise At ABW

  24. #24
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    Talking
    After the first 19 columns posts, I think this thread have become inconsistent too.




  25. #25
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    lol

    ok, admit I am sufferin from "Can't Remember Schitt"
    a lot more lately lol

    teddy bear overload

    That - and being a Yankee

    ok, guilty on both counts

    Teddy bear reminds me, I found a vampirebear on twitter and immediately thought of Stinky lol He isn't leading a double life is he......one as a skunk and one as a bear?

    hmmmm, really have gotten this thread off topic at this point. lol

+ Reply to Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Integrating Linkshare
    By bbrooksfineflowers in forum Starting an Affiliate Program & Merchant Q&A
    Replies: 1
    Last Post: December 7th, 2012, 11:58 AM
  2. Need some help integrating AssocTRAC
    By agonzalez in forum Affiliate Marketing Tracking Software and Networks
    Replies: 2
    Last Post: February 22nd, 2007, 03:53 PM
  3. Need some help integrating AssocTRAC
    By agonzalez in forum Marketing Resources & Power Tools
    Replies: 0
    Last Post: February 14th, 2007, 04:57 PM
  4. Please help: integrating text links
    By AffilNow in forum Commission Junction - CJ
    Replies: 2
    Last Post: July 14th, 2004, 09:27 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •