Results 1 to 24 of 24
  1. #1
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Broken Datafeeds (zip files with gz extension)
    Argghhh, so first GAN decides to convert to zip instead of gzip for no good reason, then they get the conversion process totally wrong by:

    1) Assigning some of the new zip files the wrong file extension (gz)

    and

    2) Leaving a mixture of gzip and zip files in my ftp folder

    So now I've got a mixture of zip, gzip, and zip masquerading as gzip to deal with.

    THANKS!
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  2. #2
    Grandma broke her coccyx! Uncle Rico's Avatar
    Join Date
    May 8th, 2007
    Location
    North Carolina
    Posts
    2,238
    I see a few files with a .txt extension but are clearly unreadable as well.

  3. #3
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Right now I'm writing some code to check the first couple bytes of the files to see if they have the right signature for gzip, if they don't, I'm assuming they are zip instead. Not sure if it is going to work, but its better than waiting for GAN to fix their bugs.

    Here is the article I'm basing my work on:
    http://blog.somecreativity.com/2008/...mpressed-in-c/

    File signatures are here:
    http://www.garykessler.net/library/file_sigs.html
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  4. #4
    Moderator MichaelColey's Avatar
    Join Date
    January 18th, 2005
    Location
    Mansfield, TX
    Posts
    16,232
    If you're using Unix, check out the "file" command. It'll tell you what type of file something is based on signatures.
    Michael Coley
    Amazing-Bargains.com
     Affiliate Tips | Merchant Best Practices | Affiliate Friendly? | Couponing | CPA Networks? | ABW Tips | Activating Affiliates
    "Education is the most powerful weapon which you can use to change the world." Nelson Mandela

  5. #5
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by MichaelColey View Post
    If you're using Unix, check out the "file" command. It'll tell you what type of file something is based on signatures.
    Yeah, Unix has a nice history of not depending on file extensions to determine the type (e.g. reading the file signatures to determine the appropriate action).

    Unfortunately, I'm on Windows.
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  6. #6
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    I had a feeling the technology giant would snafu the transition and put all my crons to annual. Figured old data was better than no data. I wish that I had been wrong for everyone's sake.


  7. #7
    notary sojac Herb ԿԬ's Avatar
    Join Date
    January 18th, 2005
    Location
    Central/Western NY State
    Posts
    7,741
    Cool
    {whew} I was a few days late catching up my GAN datafed merchants. that's good, for once.

    but someone please post when GAN gets it together . . .

  8. #8
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by John Powell View Post
    I had a feeling the technology giant would snafu the transition and put all my crons to annual. Figured old data was better than no data. I wish that I had been wrong for everyone's sake.
    I kind of figured that there would be a period where zip and gzip co-mingled, so I coded up my new solution with that in mind, giving preference to zip file extensions when there was two with the same name.

    I did not count on them mislabeling the files...
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  9. #9
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Found another problem.... They have also re-named the files. So for example, say you expected Target datafeed file to be "Target.gz" or "Target.zip", well, now it is "Target.com.zip"
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  10. #10
    Newbie Rocketfuel's Avatar
    Join Date
    September 12th, 2005
    Posts
    48
    Hey Guys, hope im not hijacking this thread but it is datafeed related.

    I just saw in GAN that as of the 26th May all ftp feeds would be in zip format, ok well this is a huge pain in the ass cause all my zips come with the idname of the merchant like K123456 so i have to unzip each individual zip.

    To make matters worse the zip is named K123456 and inside is the datafeed in with the file name 29586758540458547457_29889574789840404_catalog like wtf? are u kidding me?

    Its bad enough all the txt files are not just sent in 1 zip, but are sent individually and then have a different file name a mile long inside the zip is just insaine.

    This has to be a cock up right? is this going to be fixed?

    Any insight on this would be appreciated
    Cheers

  11. #11
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by Rocketfuel View Post
    To make matters worse the zip is named K123456 and inside is the datafeed in with the file name 29586758540458547457_29889574789840404_catalog like wtf? are u kidding me?
    I was surprised to see the zips in that format too, but am operating under the assumption that they will not change.
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  12. #12
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    OK, so all of my zips from Performics since 5/30 are corrupt!!! Is anyone else seeing corrupt zips?
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  13. #13
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Alright, I figured out what was wrong with my 5/30 & 5/31 zip files... They are really text files with a zip extension. WTF is going on GAN?
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  14. #14
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    Anybody made any contacts with GAN other than here? They don't seem to monitor here too often.

    I found this in their support forum:
    Google Affiliate Support replied and stated:

    Our technical team is currently working towards resolution of this issue
    and expect to have it resolved by Monday, May 31st, at the latest.

    Regards,
    Affiliate Publisher Support
    Google Affiliate Network


  15. #15
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    I just now PM'd this forum's moderator. (Jamie?)

    I tried to find a support e-mail address and couldn't. Does GAN have e-mail support or phone support any longer?
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  16. #16
    ABW Ambassador CathyM's Avatar
    Join Date
    May 30th, 2006
    Location
    Torrance, CA
    Posts
    893
    affiliatesupport @ google.com

  17. #17
    Member silent_thunder's Avatar
    Join Date
    January 24th, 2009
    Location
    India
    Posts
    52
    .ZIP datafeed were also broken.Tried to download 4 feeds yesterday all of them were broken

    I spent 6 months developing a script to use .gz lol you guys just made all that work obsolete with one move...

    I still don't understand why not give the publishers a choice on compression format..


    I tried asking them about the issue and why they made the decision to switch the formats.. instead of making the logical decision to provide all the formats..

    I got the answer for my first question which was they are working on it!!

    I am sick of them, there are easier ways to make money than these guys !! Unless someone who knows affilaite marketing and been a affilaite marketer takes over GAN..this is another failed yet successful ventures by google.
    [URL="http://smartpixels.net"]Seo Services[/URL] | [URL="http://moneymakingsutra.com"]Make Money Ideas[/URL] | [URL="http://smartpixels.net"]Micro Niche Sites[/URL]

  18. #18
    Newbie
    Join Date
    September 27th, 2005
    Posts
    28
    What is the status on the feeds?

    are some feeds still corrupt?

  19. #19
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by dabomb_gent View Post
    What is the status on the feeds?
    are some feeds still corrupt?
    In my case, the feed files were never corrupt, they just had a format <-> file extension mismatch. Once I started ignoring file extensions and used the file signatures always, with three different code paths based on the true format, everything started working for me and I stopped monitoring/troubleshooting....
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  20. #20
    Newbie
    Join Date
    September 27th, 2005
    Posts
    28
    ok, I found also a new way to work with the zip files. It took me a while to find the solution though

  21. #21
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    Quote Originally Posted by isellstuff View Post
    In my case, the feed files were never corrupt, they just had a format <-> file extension mismatch. Once I started ignoring file extensions and used the file signatures always, with three different code paths based on the true format, everything started working for me and I stopped monitoring/troubleshooting....
    If you get bored in the next few days could you explain that in a "for dummies" version.


  22. #22
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by John Powell View Post
    If you get bored in the next few days could you explain that in a "for dummies" version.
    The first couple of bytes inside of a file identify what type of file it is. You can view the bytes by loading the file into a binary editor, or by using the Unix "file" command as Michael suggested, or by writing your own code, such as the link I posted previously in this thread.

    zip files should start with "50-4B-03-04" and gzip should start with "1F-8B-08". If it starts with neither of these, then I assume it is text and read the first line, looking for the first part of the GAN datafeed header. If I don't find the datafeed header, then I assume it is corrupt.

    So, in pseudo code:

    if( starts with gzip byte signature ) {
    uncompress with gzip and process
    }else if (starts with zip byte signature) {
    unzip and process
    } else {
    Read first text line (e.g. ascii text, not binary as previous file reads)
    if( first line starts with GAN column header) {
    process
    }else {
    corrupt file, abort.
    }
    }

    I'm sure there is a command line way to do this on Unix using the file command, shell script and piping the result into SED|AWK|Grep or whatever... (my unix is rusty)
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  23. #23
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    BTW, Jamie and the GAN engineering team have been monitoring this thread. If you are still having problems, you should PM the GAN moderator. I stopped bugging them because I finally got my code to work regardless of the file types they push down to my server.
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  24. #24
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    Quote Originally Posted by isellstuff View Post
    BTW, Jamie and the GAN engineering team have been monitoring this thread. If you are still having problems, you should PM the GAN moderator. I stopped bugging them because I finally got my code to work regardless of the file types they push down to my server.
    I think that I'll wait on GAN. I can't believe they would leave us to make a work around for a problem they can easily solve on their end. Thanks for the tutorial though, and I may as yet have to use it.


  25. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Using CRON job to delete zip files
    By Convergence in forum Programming / Datafeeds / Tools
    Replies: 13
    Last Post: July 6th, 2012, 05:44 PM
  2. How to FTP and Import CJ datafeeds from the CJ Zip File...
    By Mr. Sal in forum Programming / Datafeeds / Tools
    Replies: 20
    Last Post: June 10th, 2009, 04:21 AM
  3. Batch Processing Datafeeds & Index Files?
    By huhu in forum WebMerge (Fourthworld.com)
    Replies: 3
    Last Post: September 12th, 2003, 12:16 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •