Results 1 to 3 of 3
  1. #1
    Newbie
    Join Date
    February 28th, 2006
    Posts
    2
    Question Newbie
    Hi All,

    I hope this is the right forum for my question.

    I am developing a price comparisson solution but have come across a problem
    loading data from my feeds to the database. I am trying to handle a huge amount of feeds and i cant seem to efficiently parse them and load into the database.

    Is my whole approach wrong? I would highly appreciate any feedback and/or advice on this matter.

    Thx,
    Orry.

  2. #2
    Newbie
    Join Date
    December 18th, 2006
    Posts
    6
    Is your parse tripping up on some character or something? What's throwing an error?

    John

  3. #3
    Newbie
    Join Date
    February 28th, 2006
    Posts
    2
    Hi,

    Well, nothing is actually throwing an error. We have tried to bulk load data feeds in XML format to no avail so reluctantly have used text format.

    As some files are as big as 1GB we had to split them into managable pieces and process each piece.

    The process is lengthy and consumes all CPU time. creating the full text search index on those records later on takes forever.

    I am confident this is not the way to go about things. Any advice is welcomed.

  4. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Not a newbie to ABestWeb but still a newbie
    By Asima in forum Introduce Yourself
    Replies: 3
    Last Post: January 3rd, 2015, 08:59 PM
  2. Newbie Newbie Newbie
    By kimanna in forum Introduce Yourself
    Replies: 10
    Last Post: November 21st, 2012, 12:11 PM
  3. Hi From NC Newbie!
    By TKG - NC in forum Introduce Yourself
    Replies: 6
    Last Post: July 27th, 2011, 02:50 PM
  4. Greetings! Newbie Here!
    By GoNicoFish in forum Introduce Yourself
    Replies: 2
    Last Post: October 2nd, 2006, 10:38 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •