Page 1 of 4 1234 LastLast
Results 1 to 25 of 92
  1. #1
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    I can't get a 500mb file to open. What's in that file, everything on Gods green earth? [img]/infopop/emoticons/icon_eek.gif[/img]

  2. #2
    Pimp Duck popdawg's Avatar
    Join Date
    January 18th, 2005
    Location
    Take off eh?
    Posts
    3,249
    In triplicate if you have Catalog City [img]/infopop/emoticons/icon_razz.gif[/img]

    Game on!!!!

  3. #3
    Full Member Abusaki's Avatar
    Join Date
    January 18th, 2005
    Location
    Phoenix, AZ
    Posts
    246
    Same thing here,
    Can't open to convert it into a text file

  4. #4
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    551
    And there I was, thinking that I was the only one. But look on the bright side, it's updated every day [img]/infopop/emoticons/icon_wink.gif[/img]

    You can get the file in either csv or txt, just ask. I changed my 244MB unopenable csv file to a 244MB unopenable txt file.

  5. #5
    ABW Ambassador cditty's Avatar
    Join Date
    January 18th, 2005
    Location
    Memphis TN
    Posts
    1,434
    Atleast you guys have files there to get. [img]/infopop/emoticons/icon_frown.gif[/img] There is nothing there for me yet.

    Chris

    ----------------------------
    Scriptsforyoursite.com - Coming soon
    Featuring datafeed import scripts for Art.com, Coldwater Creek, OverStock.com, Buy.com, TigerDirect and more.....
    Send me a Private Message for more information.

  6. #6
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    Well I'm going to try it on my bigger pc and see what happens.

  7. #7
    Full Member
    Join Date
    January 18th, 2005
    Posts
    270
    quote:
    I can't get a 500mb file to open


    This is one of those times where bigger isn't better. [img]/infopop/emoticons/icon_smile.gif[/img]

    It's real simple to deal with. Simply write yourself a program that splits the file in to a bunch of sub-files that you can deal with.

    I'm wondering just exactly what the heck a non-programming type would do with a 500 meg raw data file other than to get hopelessly lost in the process.

    CodeJockey.

  8. #8
    Ad Network Rep ToddCrawford's Avatar
    Join Date
    January 18th, 2005
    Location
    Santa Barbara, CA
    Posts
    2,162
    Try installing more RAM. I recommend at least 1 GB. [img]/infopop/emoticons/icon_wink.gif[/img]

    Todd Crawford
    Commission Junction

  9. #9
    ABW Ambassador cditty's Avatar
    Join Date
    January 18th, 2005
    Location
    Memphis TN
    Posts
    1,434
    Somebody PLEASE tell me that all the datafeeds are not part of one file and that they are all seperated.

    Chris

    ----------------------------
    Scriptsforyoursite.com - Coming soon
    Featuring datafeed import scripts for Art.com, Coldwater Creek, OverStock.com, Buy.com, TigerDirect and more.....
    Send me a Private Message for more information.

  10. #10
    Ad Network Rep ToddCrawford's Avatar
    Join Date
    January 18th, 2005
    Location
    Santa Barbara, CA
    Posts
    2,162
    You get a feed for all advertisers that you are affiliated with in one file. As you join new programs, their catalogs are automatically included. You can parse the file to get catalogs from individual advertisers.

    Alternatively, you can request individual feeds or select a few advertisers and not get feeds from every advertiser you have joined. But, as you join new programs, you will not automatically get more feeds added. This choice is for publishers that only want feeds from a few advertisers and will not want to add more later. If you go this route and later request additional feeds, there may be additional costs to getting them set up.

    Choose wisely.

    FYI - we will be adding a feature in Q1 that will break up the one big file into smaller, more manageable files.

    Todd Crawford
    Commission Junction

  11. #11
    Pimp Duck popdawg's Avatar
    Join Date
    January 18th, 2005
    Location
    Take off eh?
    Posts
    3,249
    So CJ will be charging MORE THAN ONCE to set up datafeeds?
    Choose wisely?

    No comment at this time due to counting. Promised I would [img]/infopop/emoticons/icon_wink.gif[/img]

    Game on!!!!

  12. #12
    ABW Ambassador cditty's Avatar
    Join Date
    January 18th, 2005
    Location
    Memphis TN
    Posts
    1,434
    Todd, I have a few comments to make on your last post and unfortunately there is not anything positive in any of my comments.

    Currently I have 170 merchants. If 1/2 (85) of those offer a feed and 1/2 of those are around 200,000 lines each (from my experience with LS, that number is low), then you are looking at MASSIVE files to deal with. That would be 1.7 million lines. Some of the feeds I have seen around 600,000 lines are well over 250 meg. Unless someone is on a REALLY good server, using this file will be impossible. You said above to get more ram. How much ram would you suggest to read and handle a 1.7 million line file? Best estimates, 1.7 million lines might be around 750 meg.

    These numbers are just estimates from what I have seen in other feeds. But the sizes I am seeing amd estimating are going to make this a bad/almost bad choice to mess with.

    Sorry to go off on a rant.

    Chris

    ----------------------------
    Scriptsforyoursite.com - Coming soon
    Featuring datafeed import scripts for Art.com, Coldwater Creek, OverStock.com, Buy.com, TigerDirect and more.....
    Send me a Private Message for more information.

  13. #13
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    532
    Not only am I still waiting for a CJ datafeed (almost a month now)...I'm also still waiting for ANY of these major networks to get a clue and realize that datafeeds should be treated just like any other creative...FREE and easy to access (IMHO).


  14. #14
    Pimp Duck popdawg's Avatar
    Join Date
    January 18th, 2005
    Location
    Take off eh?
    Posts
    3,249
    I agree.
    Currently my unzipped datafeed from the last date I received it was 956mb.
    That does not include the 15 or so merchants I have joined in the last 12 days or so and the merchants that I plan on joining in the future.
    1) Something has to be done about the bloat in these feeds.
    I am fairly certain that Catalog City does not actually have 2,909,817 active product links. Yes that is the biggest example and the worst but there are many others that have a serious problem with bloat.
    2) So if the fee is to setup the feeds, and an ftp or delivery service, why would there be additional costs? It's setup. Even linkshare charges once and I think someone mentioned the same at Performics but in all honesty I am not 100% certain about them.
    3) Again I don't mind paying for a feed, but it seriously has to done right otherwise what's the benefit to either publishers or the merchants? We end up advertising out of stock items that frustrate the customer so they don't buy. Nothing frustrates me more when I click on something advertised and it's not there.

    Clean feeds in consistent formats that actually contain product that works. That's all I want for Christmas [img]/infopop/emoticons/icon_smile.gif[/img] Oh, with a side of the freeness that was implied when this was originally discussed.

    Game on!!!!

  15. #15
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    Well I was able to open it with my 1.5GBs of ram though it is still a ten minute process. I can upload it into mysql now that I know what the structure of it is and I wrote a php script for printing out all the goodies.

  16. #16
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    Ah here's a clue. Turn up your virtual memory file size on your hd. I was able to ad another 12GBs and now can open the file in about 5 seconds.

  17. #17
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    Argentina
    Posts
    513
    Use a real viewer that do not need to open the entire file, like DOS ones, DN (DOS navigator) opens 500mb files for view with 2mb RAM in less than 1 second, NC would work fine also, you can run DN in a MSDOS window, or get a real viewer for windows.

    Once you know the structure, you can easily convert the file to a database using a sql server or powerful engine.

    just for experienced programmers.

    Who designed the datafeed generator at CJ? that guy really needs to learn about something called user...

    Fer

  18. #18
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    1,916
    Todd/CJ - Thanks for getting this going! Sounds like some bugs to work out but.. thanks.

    I have no problem with big files but these seem quite large [img]/infopop/emoticons/icon_smile.gif[/img] Seems like a lot of bandwidth being used as well.

    Not possible to have a feed control panel with a list of all advertisers with [ ] selection boxes to enable?

    Just some thoughts, thanks again. Good Luck. [img]/infopop/emoticons/icon_smile.gif[/img]

    --
    The Newer Nicer Joseph [img]/infopop/emoticons/icon_smile.gif[/img]

  19. #19
    Crazy Cat Lady Heidi's Avatar
    Join Date
    January 18th, 2005
    Location
    Rochester, NY
    Posts
    1,685
    after its been requested and your updated to the new feed -- how long does it take to get it? I'm still waiting -- just keep getting 404 error messages.

    Heidi
    Fit2a-t - Make Money Selling T-Shirts From Your Site!!

  20. #20
    ABW Ambassador CrazyGuy's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,463
    Todd

    This clearly needs to be re-thought. It would appear that the priority given to streamlining the process may render it useless.

    It seems such a no-brainer that there is only one workable solution - publishers choose which advertisers product catalogs they want, and they are provided with a fixed filename/location per advertiser/publisher. Then we can look at file dates to see what has really been updated, and transfer accordingly.

    These files are going to be really massive in some cases - and publishers may have different destinations for different parts of the file contents. Transferring a file to one server to strip out the parts to send to other servers is not very efficient, especialy when only parts of the file are truly dynamic (I'm guessing here, but I don't believe all product catalogs are going to be (or need to be) updated as frequently).

    I speak as someone who hasn't seen this in action yet. I've held off as the previous set-up seemed pretty messy. The advice to "contact Technical Services by selecting 'ask a question' from the 'Support' drop-down menu, then selecting 'data transfer' and 'product data feed' is kinda snookered by the fact that those options don't exist. [img]/infopop/emoticons/icon_rolleyes.gif[/img]

    So I'll shut up until I see it for myself ...

    Are you Crazy?

  21. #21
    ABW Ambassador webmarm's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,713
    OOOOOFFFFFFFFFFFFFFFFF!!!!!!!!!!!!

    Tried increasing my virutal memory (2GB), still can't finish opening the file. 512MB RAM was my mistake. I thought that would take care of all my needs for a couple of years.

    OOOOOOOOFFFFFFFFFFFF!!!!!!!

    - - - - -
    42. Yup, the answer to life, the universe, and everything.

  22. #22
    ABW Ambassador iucpxleps's Avatar
    Join Date
    January 18th, 2005
    Posts
    648
    You guys are lucky I'm on a dial up..I downloaded a 16mb file and it gave me an unexpected file end error..And it said 2.8GB file size unpacked in winrar status! next day file got updated and it's size is now 60MB. I cant download and u/l this file everyday in order to make my sites dynamic..it's a joke! So I have to manage this file on my server. and I tried this but the txt.gz file didnt want to be unpacked with an error: invalid compressed data--format violated.

    I thought It was a mistake for that file but next day(today) the updated 60mb file is still giving the same error. And I cant download 60mb in order to test it on 56k..

    This is the info what I get from file command on linux:

    bash-2.05$ file bla.txt.gz
    bla.txt.gz: gzip compressed data, deflated, last modified: Wed Dec 31 19:00:00 1969, os: MS-DOS

    1969 MSDOS? LOL!!!!

    I'm using ask a q and currently waiting my last response about this from Craig..I dont know the problem yet but I know that I would not use this thing if I have to download/upload it everyday to my computer on dial up..It should open on my server and if this error is about ram sizes next thing I'm gonna do is to send a little message to Tom hehe..

    ooh well..

    He who steals a minaret prepares a proper
    cover beforehand, said of someone who intends to do something illegal.

  23. #23
    Full Member
    Join Date
    January 18th, 2005
    Posts
    266
    Who serves the images for the product data feeds? Still the merchants or the publishers?

    Tami

  24. #24
    Full Member
    Join Date
    January 18th, 2005
    Posts
    226
    Everything was great until some brainiac decided to fix it.

    All you used to have to do was send an email off to prod and the feed would magically appear. and all parties benefited.

    Now you have to ask a question stating catagory and sub-cat, use a specific title for your question and the hope to god someone answers. Then if the person that you are communicating with doesn't get bored half way through the feedback phase and starts ignoring you, you may if you're really lucky, get a file that you can't open. Great.

  25. #25
    Ad Network Rep ToddCrawford's Avatar
    Join Date
    January 18th, 2005
    Location
    Santa Barbara, CA
    Posts
    2,162
    OK - News Flash on the Data Feeds.

    I hate to break it to you but you aren't supposed to open the catalog feeds. They are designed for publishers with databases and the FTP feed should simply be consumed into the database. Once it is in your database, then you can them manipulate it.

    You will always have problems opening large files with you PC.

    Todd Crawford
    Commission Junction

+ Reply to Thread
Page 1 of 4 1234 LastLast

Similar Threads

  1. Featured: [SAS] CleanSpirited.com Eco friendly, Green, and Environmentally Friendly Apparel
    By Jon Nunez in forum AffiliateManager.com
    Replies: 0
    Last Post: June 3rd, 2013, 10:24 AM
  2. Google Affiliate Network Launches User-to-User Forum
    By Google Affiliate Network Moderator in forum Google Affiliate Network - GAN
    Replies: 1
    Last Post: January 21st, 2009, 12:24 PM
  3. Three Level Hierarcy from One Datafile?
    By 1CNS in forum WebMerge (Fourthworld.com)
    Replies: 1
    Last Post: August 31st, 2003, 08:47 PM
  4. Try to load a datafile into MySQL....
    By TJ in forum Programming / Datafeeds / Tools
    Replies: 26
    Last Post: February 7th, 2003, 06:45 PM
  5. no datafile in last few days?
    By addall in forum Commission Junction - CJ
    Replies: 18
    Last Post: December 20th, 2002, 10:54 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •