Results 1 to 10 of 10
  1. #1
    Newbie
    Join Date
    January 18th, 2005
    Posts
    1
    Unhappy Not fun anymore
    I have to admit, working with the allposter script has become my least favorite thing to do in life. I did not even use the last allposter script update because of this, and it seems worse this time than the last time I fooled with it.

    I have been fooling around with this now for 4 hours and its still going strong. It took over two hours just to read the 4 data files which the information is stored in. I am now in the page generation mode and it has not shown any signs of stopping anytime soon, all the while slowing my system down to a snails pace. It will take all night long to upload the more than 10 thousand html files generated for my on-line store.

    Can't this be made into a dynamic generated script by now? Spending over 24 hours for this every three months is just a pain.

    Dennis

  2. #2
    Full Member
    Join Date
    January 18th, 2005
    Posts
    379
    It's not any easier with server scripts. Their feed has become so large it's almost unmanageable. If I remember right one of their tables (category relationship ?) has over 2 million rows in it.

    I think the way of the future will be to use their web services and forget about creating the pages on your computer or uploading the feed to your server.

  3. #3
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    Not to brag, just to let you know what kind of system will do this in a reasonable amount of time:

    1 GB RAM on my PC + Pentium 4 3.06 Ghz with multithreading and RAID striping= about 40 minutes to run the script (no multitasking though...let the script have all the resources, while you go read a book). Zipped files uploaded over cable connection = only a few minutes for 10k files. Unzip on server side = a couple of seconds.

    Trying to deal with huge datafeeds and thousands of files with a less-robust system is a pack of frustration no matter what script you're using. Doing it all on the server-side usually wouldn't be a help--often, servers have LESS power than PCs! A gamer's computer like mine will whomp all over a lot of standard-level servers, not only in RAM but in processor speed. It's well worth the investment to get the fastest thing your money can buy, even if you never play games--you won't be held up by speed issues for years, and can throw practically anything at it (although there are still some limits). I've probably gotten more good out of it for NONgame uses like running feeds, than for games. (Not that I don't give it plenty of use for games, but they'd probably run fine with half the speed! But some of my feeds would take ages...)
    Last edited by Leader; April 26th, 2005 at 07:06 PM.
    There is no knowledge that is not power. ~Hemingway

  4. #4
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    I would love to spend 24 hours doing this more frequently!!! Meaning would like more frequent data updates. I run a new GoCollect site with the GoCollect script AT LEAST once a month, and it takes several hours to build and about 24 hours to ftp.... But I make sales by keeping the site fresh....
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  5. #5
    Full Member
    Join Date
    January 18th, 2005
    Posts
    379
    24 hours to ftp
    That's just a silly amount of time. Are you on dial-up ?

    Do like leader says and tar.gz locally and then ftp one file and then unzip it on your server. If your host is Cpanel based I thing you can unzip it in the file manager.

  6. #6
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    Quote Originally Posted by MrSpeed
    That's just a silly amount of time. Are you on dial-up ?

    Do like leader says and tar.gz locally and then ftp one file and then unzip it on your server. If your host is Cpanel based I thing you can unzip it in the file manager.
    I'm on cable, and I don't like ftp'ing the zip file, I like doing it my way. And it isn't actually 24 hours, I just let it run overnight and sometimes cable crashes and I have to restart it in the morning. No big deal.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  7. #7
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Why not FTP a zip file? It's much easier. You need ssh access to your server to unzip it though.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  8. #8
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    I have ssh access. I like to do it my way No big deal.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  9. #9
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Problem with uploading multiple files is it's difficult to know where you left off if you're accidentally disconnected. With a single file you can resume the upload after you're disconnected. Plus it's a great deal faster. I'm not so much trying to change your ways but explain to the anonymous readers which is better

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  10. #10
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    Actually, WS_FTP Pro shows me exactly what files have been uploaded as they are loading so that if there is a disconnection I know what has been done already. It also automatically reconnects. And a zip file of 60,000 pages is still a huge file that takes a long time to load, and if interupted, needed to restart all over again. so my changed pages get there sooner than by using your method.

    Neither way is "better", they are just different.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  11. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Replies: 9
    Last Post: March 21st, 2007, 12:00 AM
  2. I can't stand it anymore...
    By Rick K. in forum Virtual Family and Off-Topic
    Replies: 10
    Last Post: September 8th, 2004, 08:14 AM
  3. We're not alone anymore!
    By Billy Kay in forum Merchants opposed to ParasiteWare
    Replies: 2
    Last Post: August 6th, 2004, 10:55 AM
  4. Not a Newbie anymore!
    By northernstudio in forum Midnight Cafe'
    Replies: 6
    Last Post: September 22nd, 2003, 06:38 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •