Results 1 to 12 of 12
  1. #1
    Member tsmgroup2's Avatar
    Join Date
    January 18th, 2005
    Location
    New Providence, PA USA
    Posts
    155
    Question datafeed transfer into a database
    The easiest and quickiest method to achieve this would be what? What form must the datafeed be received in for easiest transfer? What if I receive the datafeeds by email or webmail thru my site?

    Thanks

  2. #2
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    Upload the file to your database directory. Then, for MySQL (and, for a tab-delimited file):
    (command line)
    LOAD DATA INFILE "filename.txt" INTO TABLE tablename;
    or depending on your MySQL version, you may need to type
    LOAD DATA INFILE "filename.txt" INTO TABLE tablename fields terminated by '\t';

    Easy peasy...

    There's ways to script it, too, but the above is so easy that I just keep doing it that way.

    As for the form of the file, I like tab-delimited, but some people here swear by pipe-delimited. Personally I tend to swear AT pipe-delimited (makes it hard to work with in spreadsheets. Spreadsheet manipulation can come in handy for the little stuff...).

    CSV (comma-seperated) is considered a pain, probably by all. Try for a more "oddball" delimiter, one that will NOT appear in the data itself! But, lots of indies and outside feeds come in this format.
    There is no knowledge that is not power. ~Hemingway

  3. #3
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    You can also check this thread:

    http://forum.abestweb.com/showthread...proof+datafeed

    It's a pretty detailed discussion about importing datafeeds into mySQL.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  4. #4
    Member tsmgroup2's Avatar
    Join Date
    January 18th, 2005
    Location
    New Providence, PA USA
    Posts
    155
    Thank you Leader and Snib!
    May I ask one other thing? What is the best method of receiving the datafeeds for direct processing into my database?

  5. #5
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Quote Originally Posted by tsmgroup2
    Thank you Leader and Snib!
    May I ask one other thing? What is the best method of receiving the datafeeds for direct processing into my database?
    Personally I'll accept either httpd or FTP. I have a system for retrieving either on my server. One thing that bugs me is merchants who try to send the datafeed via email. I can't automatically parse email attachments. Another that I don't like to use is FTP transfers to my server. I like to disable FTP for security purposes, so merchants who can only send the file to my own FTP server are out of luck.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  6. #6
    Member tsmgroup2's Avatar
    Join Date
    January 18th, 2005
    Location
    New Providence, PA USA
    Posts
    155
    Thank you,
    Scott.
    I haven't covered how to use httpd yet in either php or mysql, can you inlighten me a little bit on how to use that? I hate asking to be led around like a little child, but what else can I do, I didn't have time or money to go to college when I was younger and now am stuck begging for help. Anyway, I appreciate your input and could definitely use any help on these issues.
    Thanks again.
    Mark

  7. #7
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    I suggest using wget via PHP to download through httpd. You're basically executing a command line application that downloads files from the web. Once the file is saved locally, you can write your PHP script to read it and add it to your database.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  8. #8
    Member tsmgroup2's Avatar
    Join Date
    January 18th, 2005
    Location
    New Providence, PA USA
    Posts
    155
    My thanks and blessings to you once again Scott. (sorry, just wanted you to feel how appreciative I am for the help and the fact that is that time of year - close to the holidays too) That detail has enlightened me and pointed me in some interesting directions for progress.
    May I ask, will my database understand the zipped files that I receive in .txt form? If not, should I stick with .dtd ?
    Mark

  9. #9
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Well if you get a zipped datafeed you need to unzip it. I usually just call unzip or gunzip via PHP to get the job done. Just make sure you use the proper parameters to ensure the unzipped file goes to the proper directory.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  10. #10
    Member tsmgroup2's Avatar
    Join Date
    January 18th, 2005
    Location
    New Providence, PA USA
    Posts
    155
    One more thing, scott. I have phpmyadmin available thru Globat, but looking at mysql and php individually, looks easier to set them up thru separate programming than thru site administration. Is this what you see?
    mark

  11. #11
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Quote Originally Posted by tsmgroup2
    One more thing, scott. I have phpmyadmin available thru Globat, but looking at mysql and php individually, looks easier to set them up thru separate programming than thru site administration. Is this what you see?
    mark
    I'm not sure what your question is here. Can you please restate it? I like phpmyadmin for creating and indexing databases. I usually only run it on my local computer though because after that I simply copy and paste the SQL code into the mysql prompt via ssh on my server. Everything else including inserting data from datafeeds is done through PHP scripts that I write myself.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  12. #12
    Member tsmgroup2's Avatar
    Join Date
    January 18th, 2005
    Location
    New Providence, PA USA
    Posts
    155
    Well, my hosting service has me setup my databases manually thru there phpmyadmin. Do you think this is a waste of time using this feature? Or do you think I should just stick with using php and mysql in my editors? If you have any better ideas, I am all ears. What ever the simplest quickiest way of getting my feeds up and running on my site is my goal.
    Mark

  13. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. How to put CJ datafeed into database?
    By oxplode in forum Commission Junction - CJ
    Replies: 0
    Last Post: April 20th, 2014, 03:03 PM
  2. Datafeed database structure
    By doobe01 in forum Programming / Datafeeds / Tools
    Replies: 6
    Last Post: September 8th, 2011, 05:05 PM
  3. zip file datafeed file transfer into database
    By rahulcbr in forum Programming / Datafeeds / Tools
    Replies: 13
    Last Post: August 28th, 2007, 04:59 AM
  4. Datafeed Database Fields
    By goofyhumor in forum ShareASale - SAS
    Replies: 6
    Last Post: February 25th, 2005, 10:12 AM
  5. datafeed/database manipulation
    By ToughTurkey in forum Programming / Datafeeds / Tools
    Replies: 0
    Last Post: February 25th, 2004, 01:48 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •