Results 1 to 11 of 11
  1. #1
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Need Programmer for FTP script, datafeed import script
    OK, folks, I'm at the end of my rope. I know I'd asked this before, but let's just start from scratch.

    I am trying to get ShareASale datafeed data into my database.

    (1) Merchants have granted me ftp access, and now I need to get the data.
    (a) When I manually download data, apparently SmartFTP is doing something wrong (even after I disable keep-alives), and my IP gets blocked by SAS.
    (b) In any event, I need some kind of script that will check for updates and download only the new versions of datafeed files.

    I've tried a variety of strategies, and basically I am NOT INTERESTED in "learning how to do this" -- I don't want to learn PHP or how to do this with VBscript or Python or whatever. I want to pay someone to set this up for me.

    (2) I have the datafeed files, but my efforts to create a working "import script" (including hiring and paying two different programmers) have been a huge waste of time; I now have a script that crashes during each file load, and when it does manage to download a sequence of rows from the datafeed file, it does so at a pace of just 5 rows per second (which means it will take many DAYS to import a group of large datafeed files).

    Yes, I know there are lots of piece-meal example scripts out there, mostly in PHP, but I can't make sense of them. Yes, there are lots of different tools, including Microsoft's SSIS/DTS (which aren't included in Microsoft's express-edition products and may or may not be included in the developer edition). Thre are third party products like teratrax (?) which claims to do some of this, but I can't get them to work on my system. No, I don't want to learn how to do this myself -- I want to pay someone to set this up for me.

    So, what I am looking for are: (A) People who write to me at markwelch@markwelch.com and tell me that they are able and willing to do this work, and how to contact them to talk about the project, costs, etc. or (B) Referrals to people you know who have done this exact work before, who may be available to hire.

    I cannot figure out any way to actually provide remote access to the computer I am using, so that's another HUGE issue. This may mean that I can only work with someone in the San Francisco Bay Area; or maybe someone can figure out a way to get around the McAfee firewall without completely opening my system to all invaders. (No, PLEASE don't send me isolated suggestions -- I want to hire someone, period.)

    I've been working on this full-time for several weeks, and I am really, really tired of having dozens and dozens of people tell me, "you'll be able to figure this out, just try a little longer." I am hitting all kinds of problems, and each time I encounter them, I do searches and find out that these are "commonly known problems" that I shouldn't be having trouble with since I should be an experienced programmer in VB/PHP/SQL/ADODB/etc etc.

    I have a Dell computer that uses Windows XP MCE, and I need scripts that will run on this computer.

    I really would like to understand this, so it would be best if the scripts/programs were comprehensible -- something I can make sense of and perhaps modify myself in the future.

    At this stage, I am not even looking for any kind of data "cleanup" or "transformation" -- I just want to get the raw data into a database table. (There are some major cleanup issues, of course, including removing HTML and non-working internal and external links from some datafeeds, but that's something for the next stage, which can't even start until I have a valid database table with real data.)

  2. #2
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Why are you downloading the datafeeds to your local computer? It's much better to have your web server do the downloading while maintaining the database there. Putting your local computer in the mix will create a huge bottleneck in the long run.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  3. #3
    Pimp Duck popdawg's Avatar
    Join Date
    January 18th, 2005
    Location
    Take off eh?
    Posts
    3,249
    I was about to say you should talk to Scott, but ... there he is already
    ================================================================
    Been away, now I'm back. Not as much, but I'm back & starting from scratch. Where I was, was fantastic. Where I am now, less so. Things have changed, become harder. So have I. Game ON!!!
    ================================================================

  4. #4
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    The SQL Server database will reside on a computer in my office, not on my web server. I am not planning to serve pages out of the SQL Server database, nor to draw "live" data from the SQL database when serving web pages. And my current dedicated server hasn't got that much power.

  5. #5
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Wow, I had expected more of a response to this -- not a single reply from anyone.

  6. #6
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    So you just need a program to get the data into your SQL database? You don't need something to generate pages from it? Also, do you already have the database structure? To be honest datafeed work is extremely extensive and can go on and on for weeks, months and years. So a job like this can be intimidating unless it's got a clear end to it.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  7. #7
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    Talking
    Quote Originally Posted by markwelch
    Wow, I had expected more of a response to this -- not a single reply from anyone.
    Well, maybe it was because of the on your post!

    No, I don't want to learn how to do this myself -- I want to pay someone to set this up for me.
    and........
    (No, PLEASE don't send me isolated suggestions -- I want to hire someone, period.)
    and...........
    The SQL Server database will reside on a computer in my office, not on my web server. I am not planning to serve pages out of the SQL Server database, nor to draw "live" data from the SQL database when serving web pages. And my current dedicated server hasn't got that much power.


    so... .

  8. #8
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    Update: I've just interviewed a programmer for this project (located through CraigsList.org). I'm probably going to hire him, and once again I will spend a LOT of money (the third time!) to try to tackle this phase of the project. Like the earlier folks, he has never worked with datafeeds before, but it's pretty clear that I'm not going to find anyone who has that experience, to do this work.

  9. #9
    Newbie
    Join Date
    January 4th, 2006
    Location
    Berlin
    Posts
    41
    You will find the people with experience to do the work, but most people with the experience do not need the work if you see what i mean,

    If you havent started with the other guy, tell me what kind of money we are talking about and i probably can loan one of my guys onto the task.

  10. #10
    ABW Ambassador netnow22's Avatar
    Join Date
    January 18th, 2005
    Location
    Columbia, SC
    Posts
    748
    Before you spen all that money, Talk to Roman at etnasoft.com. He is very good and designed and programmed my site below.

    Be sure to tell him Jason Bishop sent ya, and you will not regret it or waste money. I went through twenty diiferent programmers and wasted tens of thousands of dollars before i found Roman

  11. #11
    ABW Ambassador netnow22's Avatar
    Join Date
    January 18th, 2005
    Location
    Columbia, SC
    Posts
    748
    If you have a dedciated server, you should be able to use it. WHat kind of dedicated server do you have. I have a windows 2003 2.8 single processor and it works great handling over 1 million products

  12. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. SAS/FTP Datafeed Import Script
    By mobilebadboy in forum Programming / Datafeeds / Tools
    Replies: 36
    Last Post: March 7th, 2009, 08:33 PM
  2. Need Programmer for FTP script, datafeed import script
    By markwelch in forum Programming / Datafeeds / Tools
    Replies: 0
    Last Post: March 5th, 2007, 04:24 PM
  3. Non-PHP script to import datafeed to database
    By markwelch in forum Programming / Datafeeds / Tools
    Replies: 5
    Last Post: February 12th, 2007, 06:12 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •