Results 1 to 10 of 10
  1. #1
    Full Member
    Join Date
    January 18th, 2005
    Location
    UK
    Posts
    273
    I have several big feeds from different schemes which are a bit too big to use asis via php.

    My thought is to split each big one up into a number of smaller ones.

    If this was a one off exercise then it is dead simple to open the big feed in Excel and then use multiple Excel data/filter operations to get a series of subsets, copying and pasting each as a new feed file.
    I have recorded this as a macro and it seems feasbile to write some VBA based on the macro code to automate the process, with a worksheet containing the filter info and the o/p file name.

    However I am a great believer in not re-inventing the wheel, so has anybody got a better way.

    I have tried looking on Google for file split software products but couldn't really find one that did what I wanted.

    Les

  2. #2
    Affiliate Miester my2cents's Avatar
    Join Date
    January 18th, 2005
    Location
    far far away....
    Posts
    2,161
    dig around hear at ABW... this has been talked about a lot..

    Joe

    And, that's the bottom line because it's my2cents!

  3. #3
    Full Member
    Join Date
    January 18th, 2005
    Location
    UK
    Posts
    273
    Well I've been digging around and can't find anything useful, apart from the fact that Andy R. has something, but I can hardly ask as my planned superstore pages have been sitting on the back burner for a while.

    I think I will do something myself with VBA and Excel, then I can add in some more code so it can create local files and do teh FTP upload automatically.


    Les

  4. #4
    Newbie
    Join Date
    January 18th, 2005
    Posts
    36
    if you have access to a linux box you can use csplit to split files, or gsplit 1.8 for windows will do the trick.

  5. #5
    ABW Ambassador CrazyGuy's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,463
    If the feed's so big it's hard to handle, what the h*ll is it doing on your PC anyway?

    Why not use php to split it up on the server?


    --------------
    Are you Crazy?

  6. #6
    Full Member
    Join Date
    January 18th, 2005
    Location
    UK
    Posts
    273
    It's not too big to handle on my PC.

    But it is too big to put on my host shared server to churn through sequentially for every single page served.

    Les

  7. #7
    Full Member
    Join Date
    January 18th, 2005
    Location
    UK
    Posts
    273
    Gsplit does not do what I want, which is basically to filter the files to different o/p files based on the content of a particular field e.g category.

  8. #8
    I like traffic lights
    Join Date
    January 18th, 2005
    Location
    Southern hemisphere - away from Fukushima
    Posts
    2,936
    One word.

    grep

    ****************************
    George Orwell got the date wrong

  9. #9
    Member
    Join Date
    January 18th, 2005
    Posts
    142
    I get the CJ datafeed containing dozens and dozens of merchants.

    I import the feed into an Access database where I use queries to split the database into one table for each merchant.

    I then import the Tables straight into SQL Server on my server.

    Problem with the CJ datafeed is it contains all my merchants, including those whose product catalog I don't use. This way I only upload the merchants I do want, I can also do any manipulation first (for example, changing '&' to 'and' in AdvertiserCategory).

  10. #10
    Full Member
    Join Date
    January 18th, 2005
    Location
    UK
    Posts
    273
    Thanks SeventiesMartin

    Thats sounds like the best plan as it gives other advantages, but being a bit impoverished in my case it will have to be mySQL and not SQL Server.

    Les

  11. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Replies: 19
    Last Post: May 12th, 2013, 12:09 PM
  2. Splitting the Value of a Page in the SERPs
    By Kevin in forum Midnight Cafe'
    Replies: 1
    Last Post: April 7th, 2009, 06:19 PM
  3. splitting datafeeds into smaller feeds
    By dflsports in forum Programming / Datafeeds / Tools
    Replies: 25
    Last Post: February 24th, 2008, 11:06 AM
  4. Splitting the Fields in Overstock Feed
    By Rogi in forum Programming / Datafeeds / Tools
    Replies: 9
    Last Post: September 2nd, 2005, 02:59 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •