Results 1 to 17 of 17
  1. #1
    ABW Ambassador kse's Avatar
    Join Date
    November 29th, 2005
    Posts
    2,511
    Question Reading Data Feed from txt, CSV, MS-Accees or SQL??
    I have a Product Search on my site that has a total of over 100,000 items from all merchants. Right now I am reading the feeds on my site from a different txt file for each merchant. It takes around 13 sec to search all the txt feed files. Would I be better off (and faster) to uses either CSV, ms-accees or sql to read my data from? If so which would be better??

    Thank you
    kse........

  2. #2
    ABW Ambassador
    Join Date
    November 25th, 2005
    Posts
    639
    kse,

    As you are discovering, it is not advisable to try to read live data from flat text file of that size. I recommend that you parse it into an SQL or MySQL format and then you will have several options that can speed up your searches.

    CanadianDave

  3. #3
    ABW Ambassador kse's Avatar
    Join Date
    November 29th, 2005
    Posts
    2,511
    Hi CanadianDave,
    Never used SQL, but have used txt and ms Access, do you know of any good sites on learning how to create & read mysql files?

    Thank you
    kse.......

  4. #4
    ABW Ambassador
    Join Date
    November 25th, 2005
    Posts
    639
    What are you currently developing in? (I work in a mix of ASP and PHP) That is the server side script that will actually talk to the database. It seems that PHP is most often the tool of choice for many around here...

  5. #5
    ABW Ambassador kse's Avatar
    Join Date
    November 29th, 2005
    Posts
    2,511
    I am using ASP my hosting company support both My SQL & MS SQL

    Kse.......

  6. #6
    Full Member TLE's Avatar
    Join Date
    January 21st, 2005
    Location
    Southern California
    Posts
    338
    Access will slow down significantly at 5 concurrent users, which will likely happen when the bots hit you.

    I do both MS SQL and MySQL and prefer MySQL, it's leaner and faster. You can use phpMyAdmin to administer it.

    Tuan

  7. #7
    ABW Ambassador
    Join Date
    November 25th, 2005
    Posts
    639
    TLE is right, MySql is light and easy to use.

    Check out the help for MySql here:
    http://www.mysql.com/

    I learned a lot about database scripting basics at http://www.asp101.com/. The samples are usually hooked to an Access DB but there is not too much to change in order to hook up to your MySql DB.

    Finally, save yourself the hours developing a parsing script and buy the datafeed-to-MySql script. It is only $39.99 US.
    You can get it here:
    http://www.phpmill.com/feedparser/

    Best of luck,
    CanadianDave

  8. #8
    ABW Ambassador kse's Avatar
    Join Date
    November 29th, 2005
    Posts
    2,511
    Thank you both for the information, I will start playing with mysql and will have a look at the site canadianDave suggested.

    kse.......

  9. #9
    ABW Ambassador PatrickAllmond's Avatar
    Join Date
    September 20th, 2005
    Location
    OKC
    Posts
    1,219
    Keep in mind that performance opinions are just that - opinions. If they are both the same price you might want to weigh them both and form your own opinion. SQL Server is definitely easier to work with if you are using ASP or ASP.NET.

    Once you get SQL Server optimized and you start using stored procedures you will see very good performance out of it. My SQL Server sites run rings around my MySQL sites. MySQL is probably good for a beginner, but keep your options open.

    Something else you might want to look at if you need very good DB performance (Despite which one you use). See if your hosting company can give you your database system on a different box than your website runs from. So your web server would be on server1 and your database would be on server2.
    ---
    This response was masterly crafted via the fingers of Patrick Allmond who believe you should StopDoingNothing starting today.
    ---
    Focus Consulting is where I roll | Follow @patrickallmond on Twitter
    Search Engine Marketing | Search Engine Optimization | Social Media | Online Video

  10. #10
    ABW Ambassador kse's Avatar
    Join Date
    November 29th, 2005
    Posts
    2,511
    Quote Originally Posted by patrick24601
    My SQL Server sites run rings around my MySQL sites. MySQL is probably good for a beginner, but keep your options open..
    Thank you for providing another option. Is their site that you can provide about My SQL and is their a cost for the software?

    Also the idea of having the database on another server is a great idea.

    kse........

  11. #11
    ABW Ambassador PatrickAllmond's Avatar
    Join Date
    September 20th, 2005
    Location
    OKC
    Posts
    1,219
    www.mysql.com is a good place to start but there are alot of options.

    MySQL is usually provided free or at a minimal cost with your hosting company. It should not be more than $5-$10 a month for a couple of hundred meg. What is your hosting company?

    Patrick
    ---
    This response was masterly crafted via the fingers of Patrick Allmond who believe you should StopDoingNothing starting today.
    ---
    Focus Consulting is where I roll | Follow @patrickallmond on Twitter
    Search Engine Marketing | Search Engine Optimization | Social Media | Online Video

  12. #12
    ABW Ambassador kse's Avatar
    Join Date
    November 29th, 2005
    Posts
    2,511
    I was wondering about "my sql" which is not the same as mysql or is it?
    kse.......
    Last edited by kse; August 23rd, 2006 at 08:53 PM.

  13. #13
    ABW Ambassador PatrickAllmond's Avatar
    Join Date
    September 20th, 2005
    Location
    OKC
    Posts
    1,219
    I believe it is one and the same. Which hosting company do you use and I can hop over there and read about it and help out.
    ---
    This response was masterly crafted via the fingers of Patrick Allmond who believe you should StopDoingNothing starting today.
    ---
    Focus Consulting is where I roll | Follow @patrickallmond on Twitter
    Search Engine Marketing | Search Engine Optimization | Social Media | Online Video

  14. #14
    Full Member TLE's Avatar
    Join Date
    January 21st, 2005
    Location
    Southern California
    Posts
    338
    Quote Originally Posted by patrick24601
    My SQL Server sites run rings around my MySQL sites.
    Quote Originally Posted by patrick24601
    I believe it is one and the same
    You're confusing the discussion. Might it be Microsoft(MS) SQL Server, instead of My SQL Server you were referring to?

    Tuan

  15. #15
    ABW Ambassador kse's Avatar
    Join Date
    November 29th, 2005
    Posts
    2,511
    I currently use http://www.3shost.com/ for hosting my current feed. Right now I am just collecting information and plan on starting to play with SQL after my holidays. Thank you for the offer, I may ask you a question or two after I start using mysql.

    kse....

  16. #16
    Newbie DataFeedFile.com's Avatar
    Join Date
    May 26th, 2006
    Posts
    329
    MySQL vs MSQL vs MS SQL
    MySQL is a full featured Open Source SQL server (the most popular open source SQL solution in the world)

    MSQL or mSQL usualy refers to the lite version of MySQL, I used this before in Windows. It is very light weight and used for simple database only.

    MS SQL is Microsoft SQL Server. It is not open source and known to be quite pricey.

  17. #17
    ABW Ambassador PatrickAllmond's Avatar
    Join Date
    September 20th, 2005
    Location
    OKC
    Posts
    1,219
    Tuan,

    When I said "My SQL Sites.." I was referring to MY as in the ones I own! That was a good catch - I worded that poorly.

    All -

    MS SQL Express (the lite version of MS SQL) is a free download and free to use just like some version of MySQL. Both have paid and free versions.

    Open source or not does not matter unless you actually plan on getting in anc modifying the source code to your database server. I'd be very surprised if anybody here is doing that.

    What should matter most is cost and performance, and those two priorities might flip flop depending on who you are. Don't pick something because it is free, open source, proprietary, or what CNET tells you. Picks something that solves your pain.

    Patrick
    ---
    This response was masterly crafted via the fingers of Patrick Allmond who believe you should StopDoingNothing starting today.
    ---
    Focus Consulting is where I roll | Follow @patrickallmond on Twitter
    Search Engine Marketing | Search Engine Optimization | Social Media | Online Video

  18. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Commission Report Downloading as Txt not CSV
    By NickLion in forum ShareASale - SAS
    Replies: 1
    Last Post: December 22nd, 2008, 02:03 PM
  2. OrangeLinks CSV to SQL Database
    By offerfind in forum Google Affiliate Network - GAN
    Replies: 7
    Last Post: December 8th, 2006, 09:28 AM
  3. xml/csv data feeds
    By myref in forum Programming / Datafeeds / Tools
    Replies: 2
    Last Post: November 22nd, 2004, 10:20 AM
  4. Replies: 13
    Last Post: March 31st, 2004, 03:36 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •