Results 1 to 4 of 4
  1. #1
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    Downloading Feed With CRON
    I have quite a few feeds that are on auto pilot with cron. My setup just pulls the feed file and writes over the old one. That works great till you get one that's gone sour, and it gets in your database you see that it breaks the site.

    Is it best practice to code it so that you always have one good copy to fall back on. Datafeeds are great when they are of good quality, but it seems like they are rare like diamonds.


  2. #2
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    Quote Originally Posted by John Powell
    I have quite a few feeds that are on auto pilot with cron. My setup just pulls the feed file and writes over the old one. That works great till you get one that's gone sour, and it gets in your database you see that it breaks the site.
    For the ones that I have on auto pilot with cron, I have the email set up, to tell me how many products are on the new feed now, if something goes wrong with the feed, it may show 0 products added to the table, but usually the original table would not be updated, or changed...

    Another thing, I usually will not put any new merchant feed on cron, until I have manually tested that feed, a few times before...

    Is it best practice to code it so that you always have one good copy to fall back on. Datafeeds are great when they are of good quality, but it seems like they are rare like diamonds.
    I used to leave an old copy of the feed as a backup on the server, but after a while of getting to know how some of my merchants datafeed are made, now I only keep a copy of those merchants feeds that are not that reliable, because I may not update those that often if ever...

    Also, after dealing with too many different merchants datafeed, I don't think it's a good idea to have soo many different old feeds, doing nothing on the server...

  3. #3
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    Wait a minute...

    When I made the post above, I didn't knew about the reason you opened this thread, but after reading your other post:
    Quote Originally Posted by [QUOTE=John Powell
    *** FATAL ERROR *** no items found in datafeed
    Happened again exactly as in Feb 2008, but this time downloading the latest version of make.pl didn't do the trick. There is no beta either. From the looks of the log the problem started with my 8/22/2009 cron run, but just now discovered it.
    John, I thought you was running your own scripts, but since you're just running the cron, then maybe it's a good idea for you, to keep one good copy to fall back on, in case the oher script fails...

  4. #4
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    Mr. Sal, Sorry for the confusion, but it's two separate issues. The one that started this thread is DaySpring. Your first response is pretty much how I have done things with my own scripts.

    I did get an email from cron showing the normal number of lines downloaded, but later I discovered that they had decided to sprinkle double double quotes all through the categories and descriptions. It looked like someone had "" bombed it.

    The new feed wrote over the old one on download. If I would have saved the good one then at least I could show something until they sort it out. It was such a mess that I took the merchant out of the template links. I left the php files in place in case someone comes in via google they will have something.

    Like you I run it a few times manually until going with cron. At that point I may start saving a copy of the feed. This stuff happens to often.


  5. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Cron job
    By templus in forum ShareASale - SAS
    Replies: 22
    Last Post: July 29th, 2007, 03:50 PM
  2. Cron Jobs
    By Kevin in forum Marketing Resources & Power Tools
    Replies: 3
    Last Post: March 23rd, 2005, 04:17 PM
  3. Cron/Perl
    By c4 in forum Programming / Datafeeds / Tools
    Replies: 9
    Last Post: February 4th, 2003, 01:44 PM
  4. cron/crontab help
    By Pete in forum Programming / Datafeeds / Tools
    Replies: 6
    Last Post: February 25th, 2002, 07:45 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •