Results 1 to 13 of 13
  1. #1
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    I Want to Buy/License/Share Datafeed Tools
    Several times in the past couple of years, I have tried to create or adapt datafeed scripts to work for me. Last year, I spent thousands of dollars on custom programming, and in the end I simply abandoned the project when I realized how complex it was going to be (and how slowly the programmers were doing the work).

    Last fall, I was intrigued by the very useful service offered by PopShops, but they've been plagued with outages and other problems, and their recent "new features" require lots of manual work to update older pages, and their interface just won't work for the plans I would like to implement.

    So here I am: I'd like to talk to anyone who has a rock-solid system to import, update, and manage datafeeds -- with the goal of finding some way that I might license, buy, rent, access, or use your system.

    I do NOT want a basic datafeed-import script. I need "more" because I need to be able to have datafeed changes propegate "rationally" through the system. This means that if a product's description changes, or a product is dropped from a datafeed, the system can't just modify or delete the associated product record, but instead it must maintain the old (and new) data, at least until the changes have been reviewed or approved.

    My goal is to have access to a database that I can draw from to BOTH auto-generate AND "manually" generate web sites, PPC-search ads, and landing pages. If I've invested time writing custom ads or adapting product descriptions, or writing reviews or advertorials, I don't want my work to be deleted if a merchant drops the product (because the merchant may restore it in the next datafeed, or another merchant may add it).

    As anyone who has worked with datafeeds knows, there is also a great deal of "post-import-processing" required (including those pesky HTML codes, ampersand-symbol codes, and symbols improperly mapped (most commonly quote and TM symbols converted to text that appends to brand names, e.g. Calvin Kleinr and Niketm). It would be great to work cooperatively to maintain that "import processing" data collectively.

    Of course, there's more to it than just an import script and database. I'd really love to talk to someone who's also done some API work, perhaps even using APIs to exchange ads and performance data with Google AdWords, or interacting with Amazon's web services.

    I'm absolutely open to a wide range of possible relationships for this.

    If you have something to offer, or if you'd like to talk, please email or call me. (Please don't PM me.)

  2. #2
    Beachy Bill's Avatar
    Join Date
    November 20th, 2005
    Posts
    8,266
    Hey Mark. It's a shame we didn't get to talk at Summit for a bit. While I can't address all of your wish-list, I may have something to show you in a couple of weeks to address a portion of it. Feel free to PM, email or call me to discuss further.

    - B
    Bill / Marketing Blog @ 12PM - Current project: Resurrecting my "baby" at South Baltimore..
    Cute Personal Checks and Business Checks
    If you are too busy to laugh you are too busy.

  3. #3
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    "Bump."

    I am once again looking for tools.

  4. #4
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    FYI: I have posted a new preliminary specification, explaining what I am looking for, at http://www.markwelch.com/datafeed-pr...2008-06-16.pdf

  5. #5
    Newbie
    Join Date
    January 18th, 2005
    Posts
    8
    I have looked over your specs, and I'm just trying to offer some friendly advice.
    I truly think you should broaden your specs to at least consider a system utilizing PHP/MYSQL instead of just limiting yourself to ASP. Is there any reason why you only want a system built in ASP?

    There is nothing wrong with ASP, as I have used it frequently over the last 8 years or so.... But the percentage of programmers that write ASP code in relation to PHP is very low. It may be hard to find someone with the skill level needed to build a system based on the specs that you ask for, especially one that can get enough of a grasp of affiliate marketing to build a custom system around it.

    I currently run a PHP affiliate system that I modified based on a preexisting affiliate script. I have around 100 niche based quality domains utilizing this script and all are fully developed and live on the web. The system I use is nowhere near as complicated as your specs request though, and it takes some manual work in each sites raw files before being able to turn the site on live. It works great for what I do though, and pulls in nice commissions each month. Though, I mainly look at affiliate marketing as secondary revenue to the real gold mine, selling domain names and developed websites.

    I would suggest finding an affiliate script that you like the workings of already, and then take that to a programmer. Its possible the programmer could modify the script to add the features you need, or use it as a basis to build your custom script.

  6. #6
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Quote Originally Posted by eastwave
    > I have looked over your specs, and I'm just trying to offer some friendly advice. // I truly think you should broaden your specs to at least consider a system utilizing PHP/MYSQL instead of just limiting yourself to ASP. Is there any reason why you only want a system built in ASP? // * * * // I would suggest finding an affiliate script that you like the workings of already, and then take that to a programmer. Its possible the programmer could modify the script to add the features you need, or use it as a basis to build your custom script.
    I am not excluding a non-Windows solution, but my primary goal is to have a system that I can understand, and which I can modify. Therefore, I've stated that I will pay hourly for assistance and work on a Windows solution, but I will not pay hourly for a LAMP (Linux/Apache/MySQL/PHP) solution, because I would probably not be able to understand or modify the LAMP solution without many (hundreds) of additional hours of learning curve.

    I hired two different programmers to create scripts to work with MS SQL Server; one did a fairly competent job, but ultimately I was unable to make changes to make it work consistently. My programming and database skills are just too rusty to do work without a lot of help. I'm willing to pay for the help, but only for something that I understand (or which will be fully supported at a KNOWN cost).

    If I'm going to spend thousands of dollars, I need to be confident that at the end of the work, I will have a solution that I can use for my needs. My needs are not unusual or unique -- but I am now trying to lay out the entire sequence of activities as a broad project so that I am not working with someone who can do only one tiny piece.

    Starting with an existing script would be great. I haven't yet found a script that appears "more likely than not" to be adaptable to meet my needs. I have found a PHP script that I will probably pay a few hundred dollars to have installed on a VPS for me, but I don't have a high confidence level that it will prove to be something that I can adapt for my needs -- and I am simply not willing to invest 100+ hours to bring my LAMP skills to a point where I could objectively evaluate whether the product could be adapted to meet my needs.

  7. #7
    Newbie
    Join Date
    January 18th, 2005
    Posts
    8
    Quote Originally Posted by markwelch
    I am not excluding a non-Windows solution, but my primary goal is to have a system that I can understand, and which I can modify. Therefore, I've stated that I will pay hourly for assistance and work on a Windows solution, but I will not pay hourly for a LAMP (Linux/Apache/MySQL/PHP) solution, because I would probably not be able to understand or modify the LAMP solution without many (hundreds) of additional hours of learning curve.

    Starting with an existing script would be great. I haven't yet found a script that appears "more likely than not" to be adaptable to meet my needs. I have found a PHP script that I will probably pay a few hundred dollars to have installed on a VPS for me, but I don't have a high confidence level that it will prove to be something that I can adapt for my needs -- and I am simply not willing to invest 100+ hours to bring my LAMP skills to a point where I could objectively evaluate whether the product could be adapted to meet my needs.

    You do realize that you can run PHP/MYSQL on a Windows server right?
    That may be an idea to ponder as well.

    If you are thinking of putting hundreds of domain names on one box running ASP dynamic pages showing hundreds of thousands of products, I would also be concerned on how the server will handle it. ASP creates threads which will lock up IIS on a windows webserver and can shut down all the sites on your server if you are not using separate application pools within Win2K3. I control 2 windows dedicated servers (and 4 linux dedicated servers) for my customers (and myself) and I always have to set IIS on each windows box to refresh each domain's application pool every couple of hours (especially on the sites which utilize ASP).

  8. #8
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Quote Originally Posted by eastwave
    > You do realize that you can run PHP/MYSQL on a Windows server right? That may be an idea to ponder as well. <
    Yes, I understand that in theory, I could run PHP on a Windows server. In practice, the script-writers I've contacted are uniform in refusing to try to install their products on anything other than a Linux box. Even if I found someone willing to do this, I'd still need to spend a LOT of time building my PHP skills (and "un-learning" my earlier scripting and programming skills).

    I'm not completely LAMP-illiterate -- my current web server is a Linux/Apache VPS which includes some PHP pages.

    Quote Originally Posted by eastwave
    > If you are thinking of putting hundreds of domain names on one box running ASP dynamic pages showing hundreds of thousands of products, I would also be concerned on how the server will handle it. * * * <
    You've certainly shared some good advice, but it doesn't apply to me. I don't intend to use dynamic pages at all. While I used that strategy at one time, I experienced some performance issues that led me to change back to static content pages for my sites. (In fact, I think I am currently using PHP only for my PopShops pages.)

    My usual practice now is to auto-generate static HTML pages for most of my sites. When the data changes, I re-generate the static pages (currently, I re-generate entire sites whenever there is a single change).

    In the PDF document I posted about this project, I identified "Phase Six: Generating Web Site Content Pages" but didn't elaborate. When I re-generate a new version of that document, I'll be clearer.
    Last edited by markwelch; June 19th, 2008 at 06:42 PM.

  9. #9
    Newbie
    Join Date
    January 18th, 2005
    Posts
    8
    Quote Originally Posted by markwelch
    My usual practice now is to auto-generate static HTML pages for most of my sites. When the data changes, I re-generate the static pages (currently, I re-generate entire sites whenever there is a single change). (In the PDF document I posted about this project, I identified "Phase Six: Generating Web Site Content Pages" but didn't elaborate.)

    That makes sense then, asp should work fine for you if you can find someone to code it like you want. If you are going to rely on asp to do the autogeneration of the pages I would suggest having the programmer do it where it only batches a certain amount of static pages at once, which I'm assuming is what you will do.


    I used to use asp on a mall type site with over 500,000 products in a MYSQL database, I had probably 25 domains running off the same database. I experienced some major slowdown no matter what I tried to do to optimize the database. I then switched it all over to PHP on Linux and now have around 100 sites, I also split the database up into 7 different databases based on category. The speed now is really good and there is probably close to 1,000,000 products across all the databases.


    If you do go down the dynamic asp page route again on windows I suggest installing ISAPI Rewrite which makes IIS able to MOD REWRITE pages like .htaccess files work with Apache/Linux. I used this when I was using asp and it worked well.


    Good luck in your hunt for a programmer...

  10. #10
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Quote Originally Posted by markwelch
    > When I re-generate a new version of that document, I'll be clearer. <
    FYI, I've updated the project requirements/specification document (but it's still very preliminary). I decided to simply overwrite the prior document with the new version: http://www.markwelch.com/datafeed-pr...2008-06-16.pdf (the first line says "Updated June 19").

  11. #11
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    OK, my adventure with Datafeed Studio has been an interesting diversion, but it's really, really clear once again that it's a really, really bad idea for me to try to launch a development project using technologies that are completely unfamiliar to me, and which are unsupported. (Martin provided really great support for DS, but I needed support for LAMP [Linux, Apache, MySQL, PHP].)

    Datafeed Studio, like PopShops, is a great example of the kind of foundation on which I'd like to build my solution.

    I am once again looking for developers and tools who might help me move closer to my goals, as outlined in the PDF document linked above.

  12. #12
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    I have posted a slightly more detailed version of my 'datafeed project' specifications, at http://www.markwelch.com/datafeed-pr...2008-06-30.pdf

    I'm still looking for help.

  13. #13
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    FYI: I have expanded my description of the "datafeed project" to 10 pages, and added a link to a (very preliminary) database schema.

    http://www.markwelch.com/datafeed-pr...2008-07-01.pdf

  14. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Please share what tools you use to build forms for lead gen
    By CDMe in forum Marketing Resources & Power Tools
    Replies: 5
    Last Post: October 27th, 2009, 05:32 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •