Results 1 to 6 of 6
  1. #1
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    VPS Hosting and CPU usage (throttling)
    As I've continued to try to implement a system to automate the downloading and importing of datafeeds, I've encountered yet another bizarre problem.

    I noticed last week that the Control Panel interface for eApps was often showing my CPU usage as more than 100%. That seemed strange, so I submitted a support ticket; the response was not helpful.

    I also noticed that my VPS seemed to be using only a tiny fraction of available memory. I suspect this was related, so I posted several requests for help on some forums, and (as usual) received some useless repsonses plus some cryptic suggestions. Ultimately I spent several hours tracking down the location of the my.cnf configuration file for MySQL, and then more time trying to identify the correct syntax for entries in this file, and then more time trying to identify the right values. Eventually I saw that my memory usage had risen from less than 5% to about 15-20% of available memory.

    But as noted above, this appears to have also resulted in an increase in CPU usage.

    I use a VPS (virtual private server) solution that uses LAMP (Linux, Apache, MqSQL, PHP).

    After 18 months of working "on and off" with LAMP, and with several consultants, I still consider myself "extremely unqualified" when working with LAMP. Again and again (and again and again) I find that I must spend 5, 10, or 20 hours getting a very, very basic answer (today, I spent 2 hours trying to figure out why the PHP "filemtime()" function was failing with a weird error message; eventually, the function just started to work without any changes.

    Here's the problem: When my import scripts run, MySQL pushes CPU usage to 150% or more, for several hours. After many searches, I've concluded that this is simply unavoidable: MySQL commands as much CPU as it needs, and the LAMP environment does not provide any method to "throttle" MySQL. If I am going to continue my "datafeed import project" work, then I cannot use an eApps VPS account. (I don't understand why any VPS solution would allow one account to use more than 100% of allocated CPU resources.)

    I need advice. Are there other VPS solutions that can actually throttle CPU usage intelligently? Am I really stuck with having to (again) lease a dedicated server, and this time act as the server administrator (meaning I'll need to learn much, much more about LAMP and installing modules and such)?

  2. #2
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    While editing my message, a couple of key points somehow got deleted.

    (1) The changes to the my.cnf file also appear to have also resulted in an increase in CPU usage; instead of reducing the problem, it's made it worse.

    (2) I've now received a warning email from eApps complaining that my CPU usage is persistently above 150% and is causing problems for other users. I also am seeing more "outage notices" that my own web pages aren't being served in response to requests from my monitoring services. They've offered to move my account to a "quarantine server" where it would share overbooked resources with other "problem accounts," but of course that's not a viable solution.

  3. #3
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    If you are using that much resources you will need to go to a dedicated server. A VPS is a self contained section of a server, but is not a dedicated server and your usage can indeed take down the rest of the sites on that server.

    I don't have the other answers for you, but just wanted to clarify the above.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  4. #4
    .
    Join Date
    January 18th, 2005
    Posts
    2,973
    FYI, as a very temporary work-around, I've added a sleep(1) command into the script, so that only one record is being inserted per second. This means that it would take more than 11.5 days to insert 1 million records.

  5. #5
    Full Member
    Join Date
    January 18th, 2005
    Posts
    396
    Being even a much less expert on MySQL and Linux ... I use the Linux NICE command to raise or lower my priority on the server - it works on my UnZIP utilities and keeps me from getting clobbered by my host (so far) - my server max allocation is in the range of 20-30% for less than a minute -- without NICE I can hit 50% or more --- look it up (Google) but as I remember NICE 10 = average and NICE -19 == HOG and unless you are UNIX GOD you don't get it and NICE 20 == table scraps but no one kicks you

  6. #6
    Full Member
    Join Date
    January 18th, 2005
    Posts
    396
    Also - it does appear that on the more current incantations of BASH and LINUX you can SLEEP 0.01 ... - might help

  7. Newsletter Signup

+ Reply to Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •