Results 1 to 12 of 12
  1. #1
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    TIP: Viewing large datafeed files
    I had written a long post about how I view files that are 1GB or greater in size, but then my browser crashed...

    Anyway, if you need to browse or search super big files on windows, you might try the V File Viewer. It chunks files so that you can load a 2.5GB file like the Buy.com datafeed instantly. It allows you to search within a file or within a file system and you are not limited to the current file chunk.


    Here it is:
    V - The File Viewer
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  2. Thanks From:

  3. #2
    ABW Ambassador I.M.O.G.'s Avatar
    Join Date
    February 19th, 2011
    Location
    Rootstown, OH
    Posts
    1,096
    Nice tip, I can think of a few times this could have come in useful. I have some large text files that are a bear to load up and manipulate, so I'll keep this in mind next time I need it.
    Matt Bidinger
    Online Community Engagement

  4. #3
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by I.M.O.G. View Post
    Nice tip, I can think of a few times this could have come in useful. I have some large text files that are a bear to load up and manipulate, so I'll keep this in mind next time I need it.
    It is pretty useful for browsing webserver logs or searching the webserver log folder too.

    I use log4net to log webserver errors. That is the other big file type that I have to deal with regularly.
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  5. #4
    Affiliate Manager Government Auctions AM's Avatar
    Join Date
    July 27th, 2010
    Location
    New York City
    Posts
    119
    This is great stuff! We are just setting up our data feed and this is great tool to advice for our affiliates to use!
    Last edited by Government Auctions AM; May 26th, 2011 at 03:24 PM. Reason: Typo
    Steven Shnayder
    [B][URL="http://www.governmentauctions.org"]GovernmentAuctions.orgŪ[/URL][/B]
    Join us on [URL="https://signup.cj.com/member/brandedPublisherSignUp.do?air_refmerchantid=2932365"]CJ[/URL] [URL="http://www.shareasale.com/shareasale.cfm?merchantID=10834"]SAS[/URL] [URL="http://www.governmentauctions.org/home/partners.asp"]ClickBank[/URL] [URL="http://twitter.com/GAORGAFFILIATES"]@GAorgAffiliates[/URL]

  6. #5
    Member
    Join Date
    November 3rd, 2009
    Posts
    50
    I use Programmers Notepad to open large datafeeds and logs, it works pretty well and is pretty fast

  7. #6
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by MoreBeer View Post
    I use Programmers Notepad to open large datafeeds and logs, it works pretty well and is pretty fast
    I just looked at it, and it doesn't mention chunking? How large is large? The V-Viewer tout's the ability to load 100GB files. I typically use it on anything that is > 40MB in size, otherwise I use plain old notepad.
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  8. #7
    Member
    Join Date
    November 3rd, 2009
    Posts
    50
    if I'm not wrong I've used it to open 2GB (or 4GB, can't remember) files

  9. #8
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Nice to see a datafeed thread. Anyway, for Mac and Linux users it's always good to use grep to search a file or even tail to check the format. Both are very fast.
    Hatred stirs up strife, But love covers all transgressions.

  10. #9
    ABW Veteran Mr. Sal's Avatar
    Join Date
    January 18th, 2005
    Posts
    6,795
    Quote Originally Posted by Snib View Post
    Nice to see a datafeed thread.
    I agree!

    Too bad there where not that many helpful datafeed threads back before 2005 when I need them...

  11. #10
    Newbie
    Join Date
    July 6th, 2011
    Location
    Italy
    Posts
    12
    I've got it with notepad++, it's free and I've never had a problem opening large datafeed.

  12. #11
    Member
    Join Date
    November 3rd, 2009
    Posts
    50
    Quote Originally Posted by MoreBeer View Post
    if I'm not wrong I've used it to open 2GB (or 4GB, can't remember) files
    I was wrong, maybe I confused it with another software. A few days ago I tried to use it to open an SQL dump of ~900MB and it couldn't handle it

  13. #12
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by piemant View Post
    I've got it with notepad++, it's free and I've never had a problem opening large datafeed.
    What I like about the V File Viewer is that the initial load time is independent of the file size due to the chunking. So a 20GB file will show in under a second and you can immediately start paging through chunks, or execute a search, which of course takes longer.

    Also, it has low RAM requirements, also due to the chunking. A 10GB file will consume only the first chunk's worth of RAM. It never loads the entire file into memory, so you don't have to worry about system resources.

    Oh, and the V-File viewer has the most lenient, full featured free trial period that I've ever seen. I finally bought it out of guilt.
    Last edited by isellstuff; July 6th, 2011 at 09:59 AM.
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  14. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Dealing with large datafeed files in PHP
    By MoreBeer in forum Programming / Datafeeds / Tools
    Replies: 4
    Last Post: April 6th, 2010, 07:03 PM
  2. Problem parsing large datafeed with XMLReader
    By seo london in forum Programming / Datafeeds / Tools
    Replies: 2
    Last Post: May 18th, 2007, 09:28 PM
  3. Removing HTML from Large Database Files
    By jdsguam in forum Programming / Datafeeds / Tools
    Replies: 10
    Last Post: August 10th, 2006, 09:29 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •