Results 1 to 10 of 10

Thread: Data feed is too large!

  1. #1
    Super Sh!t Stirrer SSanf's Avatar
    Join Date
    January 18th, 2005
    Posts
    9,944
    Is there any way we can get it in several smaller portions? I can't get it to open in any of my programs and it stalls my computer.

    Mr.Merchant, if you do business in any way what-so-ever with parasites, your products will not be sold on my sites!!

    Farewell, CJ! I loved you when you were young and pure. I will try to remember you that way. Disclaimer: Comments are to be interpreted as opinion unless otherwise noted.

  2. #2
    ABW Ambassador cditty's Avatar
    Join Date
    January 18th, 2005
    Location
    Memphis TN
    Posts
    1,434
    SSanf, I hate to say this, but that is becuase of the program you are using. Excel can only handle just over 65k rows. Excel was designed for number crunching and not handling datafeeds.

    If you want to handle these really large files, you are going to have to break down and get the tools that will handle it. ie...LOTS more ram, a program that can handle over 65k records, and possibly learning some sort of scripting language. I know a lot of people are using Excel for this, but that is not what it was designed for.

    I don't mean anything by this, but if merchants start making these feeds into smaller files for non-programming people to handle, then it makes it a lot harder for us programming people to manage and deal with.

    You could always look into the split command on your server and see if you can break the feeds apart.

    Just my $0.02.

    Chris

    ----------------------------
    <FONT SIZE=1>ezDatafeed.com - Merchants - Give your affiliates your datafeed outside the networks, without cost to the affiliate. Check out ezDatafeed.com for more information.
    Scriptsforyoursite.com - Featuring datafeed import scripts for Backcountry Store, Coldwater Creek, Mondera.com and many more.....</font>

  3. #3
    Affiliate Marketing Consultant Andy Rodriguez's Avatar
    Join Date
    January 18th, 2005
    Location
    Sunny Miami
    Posts
    8,384
    Ssanf,

    Email me and i'll set you up with 2 nifty programs that will do what you need....they are easy to find online too.

    AndyRodriguez.com
    Affiliate Program Management
    and Consulting Services
    P: (305) 412-3512
    C: (305) 785-4524
    E: doc@andyrodriguez.com
    W: www.andyrodriguez.com
    AIM: AffiliateDoctor

  4. #4
    Chief Betty Victor Hanna's Avatar
    Join Date
    January 18th, 2005
    Location
    San Mateo, CA
    Posts
    881
    SSanf,

    Are you referring to Excel or another program?

    You should be able to open the feed in Excel as a CSV file and if you wish, you can separate the two logical stores from each other. We have both Office Supplies and Cleaning Supplies set as two different values should you wish to break them apart. There are about 2,500 items in Cleaning and 27,000 or so items in Office.

    Let me know if that helps at all!

    Regards,

    Victor Hanna
    The BettyMills Company
    Home of the 30,000 product DataFeed
    http://www.BettyMills.com
    http://affiliates.bettymills.com/
    BettyMills ABW Forum
    800-BettyMills (800-238-8964)

  5. #5
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    England
    Posts
    4,327
    I had no problems opening the 30,000 product datafeed in Excel.
    It all depends on how much RAM you have, and which processor you have. If you have a good computer, you should have no problem using a 30,000 product file, and you don't need to know any scripting languages

    www.cjshoppingnetwork.com

  6. #6
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    Everybody talks about Excel but I'd practically stake money that Ssanf is using WORKS Spreadsheet to (try to) open this file, not some expensive officeware...

    (Not that I can say anything about that since I won't buy Excel either--seems like a royal rip to me!)

    But when it's time to "get serious" it's good to know that MySQL can open any feed.

    It is a beautiful thing, to do nothing, and then rest afterwards.~Spanish Proverb

  7. #7
    Super Sh!t Stirrer SSanf's Avatar
    Join Date
    January 18th, 2005
    Posts
    9,944
    Nope, Excel. I will see what I can do but so far, no good.

    I am starting to sus[ect that I need to add more something or other to my computer.

    Mr.Merchant, if you do business in any way what-so-ever with parasites, your products will not be sold on my sites!!

    Farewell, CJ! I loved you when you were young and pure. I will try to remember you that way. Disclaimer: Comments are to be interpreted as opinion unless otherwise noted.

  8. #8
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR> Nope, Excel. <HR></BLOCKQUOTE>

    Oops, glad I said "practically" stake money, instead of betting!

    It is a beautiful thing, to do nothing, and then rest afterwards.~Spanish Proverb

  9. #9
    Full Member
    Join Date
    January 18th, 2005
    Posts
    417
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR> I am starting to sus[ect that I need to add more something or other to my computer.
    <HR></BLOCKQUOTE>

    Come on Ssanf!
    You're making DOLLARS off those sites, INVEST in some RAM.

  10. #10
    ABW Ambassador Mike O's Avatar
    Join Date
    January 18th, 2005
    Location
    Los Angeles area
    Posts
    843
    I'd agree the amount of available RAM is likely to be the problem.

    I'm using Windows XP Home, with 1 gigabyte of RAM, and the feed opens fine for me. I just tell Excel it's in "pipe-delimited" format and off it goes.

    -- Mike

    "Men travel faster now, but I do not know if they go to better things."
    -- Willa Cather

  11. Newsletter Signup

Similar Threads

  1. Extract Data from a Data Feed
    By TalhaA in forum Programming / Datafeeds / Tools
    Replies: 6
    Last Post: February 25th, 2012, 05:40 PM
  2. Data Feed - Invalid Compressed Data in .gz files?
    By Bleddyn in forum Rakuten LinkShare - LS
    Replies: 7
    Last Post: November 29th, 2010, 09:30 AM
  3. Need help with large data feed
    By Bob_A in forum Programming / Datafeeds / Tools
    Replies: 14
    Last Post: November 9th, 2005, 11:17 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •