Results 1 to 4 of 4
  1. #1
    Full Member
    Join Date
    January 17th, 2005
    Changing my processing from local to on the Web with PHP
    I am contemplating changing my VB based preprocessing (datafeed downloading/index generation/data file formatting .../data uploading) to operate on the Web and save me the download/upload problems and time - one more step toward automating everything.

    While I believe that I can translate my VB operations to PhP, I am afraid that the cpu intensive processes will bog the web server. The vb process on a fast cpu with 4 G of memory takes 28 hours now and it does bog one side of a dual Intel.

    How do you folks that are handling large files online handle the 'cpu bogging' problem?

    Any additional comments/thoughts?


  2. #2
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Bayou Country, LA
    Quote Originally Posted by micheck View Post
    How do you folks that are handling large files online handle the 'cpu bogging' problem?
    Bogged my cpu just thinking about it all. I admire those that can rise to higher levels of complexity. Good luck on your answers.

  3. #3
    Member Prosperent's Avatar
    Join Date
    November 29th, 2009
    CO, USA
    The big thing is going to be memory usage with php, especially if the files are large enough that it takes 28 hours to process currently. PHP isn't very good at memory handling in some cases. If you can offer up more info about what it is you are doing specifically, I may be able to help.

  4. #4
    Full Member iolaire's Avatar
    Join Date
    October 3rd, 2006
    Arlington, VA
    Wouldn't converting from VB to ASP be much quicker for the code re-write?

    If you are just starting out I highly recommend looking at a "cloud" solution for processing your data-feeds. You'll need to write command and control logic so you can automatically start/stop/upload processed data etc. from your web server computer. That way all your CPU/Memory usage is on another system and will not impact your (or someone else's) web server.

    For example, I have a fairly small processing system, ~21-million products. Most of my processing is using on demand Amazon EC2 servers. Previously I kicked off four servers every other day for about 19 hours, running about $130/month in charges. Now I'm similar logic to run 1 EC2 spot instance almost full time (they can kill it if they have someone willing to pay more than me) that is running under $50/month (unix instance).

  5. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Changing my Web site to a Blog
    By bookdrawer in forum Blogging, Mobile and Social Media
    Replies: 10
    Last Post: October 11th, 2010, 12:33 PM
  2. “Local SEO” service for my web business
    By ShaneWatson in forum Midnight Cafe'
    Replies: 11
    Last Post: August 25th, 2009, 12:25 AM
  3. Php Page / Form Processing / Mysql Insert / No Page Refresh
    By Uncle Rico in forum Programming / Datafeeds / Tools
    Replies: 3
    Last Post: January 4th, 2009, 10:55 AM
  4. Web Surfers Changing Habits to Avoid Spyware
    By UncleScooter in forum Midnight Cafe'
    Replies: 20
    Last Post: July 7th, 2005, 09:44 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts