Page 1 of 2 12 LastLast
Results 1 to 25 of 27
  1. #1
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Data Feed Experts in here please:)
    Hi All,

    I was wondering if You could help me on my PHP/MySQL/DataFeed journey?

    This is what I have so far: I have PHP and MySQL on my personal PC (it's set up as local host server) and from the recommended tests it works; I have FTP access to ShareASale and ShareASale Data Feeds; I have a Spread Sheet Reader and Editor (text and html editors too); and, I also have a massive PHP And MySQL book by Larry Ullman (In other words, I think I have all the necessary componants needed to do Data Feeds)... So my question is what should I be studying or working on now?- I rather not read through this whole book to figure out what I need to do next, if You know what I mean?... So could You please give me a list of the next steps that I need to take to reach the end of my journey?; which would be the implimentation and use of data feeds on my site (so I don't waste time studying things I don't need to)?

    Thanks for Your Help and Time.

    Love You,
    HumbleFish

    P.S. Thanks to Everyone that got me this far!

  2. #2
    Not Verif-Lidated infoTim's Avatar
    Join Date
    January 18th, 2005
    Location
    Sunny Florida
    Posts
    1,021
    You'll want to get all cozy with MySQL, learing how to create a database that fits with your product feed in some way and how to write SQL queries that you'll use when you want to display a page.

    Then work on getting that feed into MySQL. I do that in PHP as well, but there's probably ways to do it differently.
    Tim
    consultant by day, affiliate by night

  3. #3
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Hi infospongellc,

    I did exactly what You suggested in another thread that I started in DW's corner of ABW; I turned my PC into a "little web server with PHP"... Thanks (see Your quote below).

    You can run a little web server with PHP on your own PC...that's a great way to learn!

    - Tim
    Is that the way You learned?

    infospongellc: You'll want to get all cozy with MySQL, learing how to create a database that fits with your product feed in some way and how to write SQL queries that you'll use when you want to display a page.
    So the above is next, ok I'll start studying.

    infospongellc: Then work on getting that feed into MySQL. I do that in PHP as well, but there's probably ways to do it differently.
    When You say "work on getting that feed into MySQL" could You go into a little more detail?

    Thanks for the Help infospongellc; and if anyone else has any suggestions I really like to hear them.

    Love You,
    HumbleFish

  4. #4
    Not Verif-Lidated infoTim's Avatar
    Join Date
    January 18th, 2005
    Location
    Sunny Florida
    Posts
    1,021
    I do all my development on one or more computers here in my office that pretty much run a mirror copy of what's on my web servers & that's a great way to learn and play around with things. You can develop a new site and test it out and experiment with things before making it "live".

    As far as getting the feed into MySQL, I don't know if what I do is typical or not, but I have some PHP programs that read each line of the data feed file and massage it into a common catalog layout or other tables that I need and that does all the SQL "insert" or "update" commands to put the data into the database. It's been a long time since I've used MySQL (I use PostgreSQL for performance and features reasons) but there might be other programs already out there that will do that.

    You're probably getting to the steepest part of the learning curve right now. Some sites like PHPBuilder.com and the on-line annotated reference manual at PHP.net are very helpful, but if you don't already have one you might want to check out one of the books that shows how to combine PHP + MySQL to make web sites...just to have lots of good examples handy.

    - Tim
    Tim
    consultant by day, affiliate by night

  5. #5
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Hi infospongellc,

    Thanks for the sites I checked out one of them and saved it to favorites and I'm going to check the other later.

    Well, I guess I not going to get the "You Put The Lime In The Coconut And Shake It All Up" response that I was kind of hoping for here... I was hopeing someone could give me a "A" you do this, and "B" you do this, and "C" You do this type of thing; but it seems that with PHP and MySQL that just don't happen or exist... It seems from the sites and books that I've looked at You kind of have to find Your own way through it?... Is that about right or am I missing something?

  6. #6
    Not Verif-Lidated infoTim's Avatar
    Join Date
    January 18th, 2005
    Location
    Sunny Florida
    Posts
    1,021
    Well, PHP and MySQL are tools, rather than solutions. There's no single hard and fast recipe for making a web site with them. That's probably where something like WebMerge comes in, the path (or railroad tracks, probably) are better established.

    Some book along the lines of "Building dynamic web sites with PHP & MySQL" might be really handy for helping you see how those pieces fit together.

    If you want detailed step-by-step instructions for making your site, well, that's getting into the realm of where I normally get $120/hr, so forgive me if I'm not sitting down and laying it all out here. :-)

    - Tim
    Tim
    consultant by day, affiliate by night

  7. #7
    Animal Lover
    Join Date
    January 18th, 2005
    Location
    oz
    Posts
    1,210
    Hi HumbleFish

    It took me a month or 2 of tearing my hair out to go from knowing zilch about datafeeds to learning mysql and php - I learnt it from a book called php and mysql web development by Luke Welling. It wasn't easy. But what I did was like what Tim said, trial and error (lots of errors - like forgetting the little semi-colons, the quotes...) - I did not read it cover to cover and am still learning. What I did do was take a feed (I used the simplest cleanest ones I could find - ones you didn't have to do much manipulation with eg the prices are all in dollars and not cents) - I think I started off with Rockler (there weren't too many columns in the table and it was a clean txt feed - no quirky tabs or weird symbols in the descriptions etc that would put things out).

    I didn't work locally, I worked off the web server...but the pages generated weren't 'live' till they worked...gradually I learnt that there were a few handy scripts which I could modify bits to and reuse over and over and if I needed to do any fancy stuff I could...I learnt as I went ie I started with doing a standard query and then jiggled with output onto a page - every time looking through that particular section in the book - I read the overview to get an idea about what it's all about and then took example scripts from the book and modified them as I went along. That's how I learnt. I don't think I'll ever be a guru but it's sufficient to get what I need on my pages and for it to be 'unique'.

    Hang in there. There's no easy way but it gets easier as you go along.

    Oscar

  8. #8
    ABW Ambassador Jane's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,350
    Well to start with A I would figure out how to upload the feed to your server.
    I use either PHPMyAdmin or MySQL Front to do this. Depending on who your hosting with you may find some info on your web hosts site.

  9. #9
    ABW Ambassador FFoc's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,015
    Here's where I started:
    PHP/MySQL Tutorial @ Webmonkey

    Pretty decent data-driven website tut.

    A good next step:
    Site Navigation with PHP @ Webmonkey

    I haven't follwed it, but you might get something out of Your First Database

    Other resources:
    http://www.phpbuilder.com/
    http://www.zend.com/developers.php
    http://www.devshed.com/c/b/PHP
    http://www.phpfreaks.com/

    And don't forget to check out http://www.php.net/links.php

  10. #10
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Thanks All,

    infospongellc, I can see how that could hamper the flow of info.

    oscar, Thanks for the encouragement... You say that you didn't read the whole book, what specific parts of your book did You study?

    Jane, I think I got the feed in there like You said; I uploaded the print view to: http://www.humblefishmall.com/database.html so you can check it out (or anyone else for that matter)... Is that what I need to have?... And is there anyway to peak at what's inside the data base or is that all I get?

    FFoc, Thanks for all the great links; I have lots of reading to do.

    And, if anyone else would like to get some questions answered in here about data feeds as long as it's pretty close to context feel free to fire away.

    Love You,
    HumbleFish

  11. #11
    Animal Lover
    Join Date
    January 18th, 2005
    Location
    oz
    Posts
    1,210
    oscar, Thanks for the encouragement... You say that you didn't read the whole book, what specific parts of your book did You study?
    I read the intro chapters and then went straight into the web development bits and picked up bits of code in there to play around with ie how to pull data out of a feed and then display it in a table.

    And then whenever something new stumped me I would go back to the book and look it up...that was how I learnt - which is probably why my code is probably not the cleanest around but as long as it works, I didn't really care.

    php isn't that hard, if you already know html, then all the bits in the echo are usually just straight html. So what I usually did was build a page in html and then whenever I needed a php script added I would call the script using a server side include.

    cheers,
    Oscar

  12. #12
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Thanks Oscar,

    Could you or anyone look this print shot of my data base and table at http://www.humblefishmall.com/database.html and tell me if I did it right; or is there a way to test it and make sure it's in there right?... I can't find anyway to get in there to see the data, is that normal?

  13. #13
    Animal Lover
    Join Date
    January 18th, 2005
    Location
    oz
    Posts
    1,210
    Why do you have so many index fields? Usually I only have the ProductID as an index and it should do. It's not critical but I thought it was a bit strange to have all the other fields as index fields as well ie the descriptions etc...

    You should be able to view the database if you have loaded it up on the server - using phpmyadmin (that's what I do) - just browse the table and take a look. It's showing me that you have 971 records in there each with the fields ProductID, Name, Link, Thumbnail, Description.

    You could probably try a simple php script to call out the data to see if it works.

    Oscar

  14. #14
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Thanks oscar,

    Go easy on me now, I don't have a clew as to what I'm doing.

    So from what You're saying a gather that I got it in there; and I should only index the ProductID... Right?

    Another question, when I load up a data feed should I edit it or should I just load it up as-is so I don't have to edit each time I update the feed?

  15. #15
    Animal Lover
    Join Date
    January 18th, 2005
    Location
    oz
    Posts
    1,210
    Go easy on me now, I don't have a clew as to what I'm doing
    We all gotta start somewhere.

    That's what I normally do - just the productID. As for editing the feed, what I usually do is download it onto my local computer, take a look at it on Excel and work out my php code from there. At the same time I clean the feed - ChrisWSO has something that cleans it for you but I haven't used it yet. That's why I suggested you pick a nice clean feed to try out first, some feeds aren't clean and by this I mean there are lots of weird spaces, tabs etc which throws the feed out when you upload it onto your server ie suddenly your description column ends up being your price column etc...

    Oscar

  16. #16
    Not Verif-Lidated infoTim's Avatar
    Join Date
    January 18th, 2005
    Location
    Sunny Florida
    Posts
    1,021
    If your feed is CSV, there's a nice PHP function called fgetcsv() which will load each row of a file into an array with the columns numbered 0...n.

    If you're using a pipe delimited file you can approximate that with fgets() and explode().

    I wrote a data import class that I sublcass depending on the type of feed a merchant has, with a mapping table for what column in the feed goes where in the database.

    - Tim
    Tim
    consultant by day, affiliate by night

  17. #17
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Thanks Ye-All',

    Well I tried to index just the productID numbers but I must me doing something wrong because all the product IDs are duplicates (everything else looks fine)?... I made a print view of how the table looks now, and I also did a print view of a table browse... Could You please take a look at them and tell what I'm doing wrong... You can see them here at: http://www.humblefishmall.com/ourtable.html and http://www.humblefishmall.com/ourtablebrowse.html ... Thanks.

  18. #18
    Animal Lover
    Join Date
    January 18th, 2005
    Location
    oz
    Posts
    1,210
    Humblefish

    You need to change the YOURUSERID and replace them all with your affiliateid from SAS otherwise you won't get credited for those sales. What I usually do with feeds is use Excel to look at them locally - I'm sure there are other more efficient ways but that's how I do it. I open the feed in Excel and then do a find/replace for my affiliateids and then when I'm quite happy it looks okay, I upload it onto the server using phpmyadmin. The table structure you have looks okay. I don't know why your productIDs are the same...theoretically if you have set the productID as the index, you shouldn't get that at all because each productID has to be unique.

    I upload the feed onto the server using phpmyadmin after I've done the cleaning.

    Oscar

  19. #19
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    You may have forgotten to set the productID field as an auto_increment field. You need to ensure that it increments every time you add a new ID. It's also important that you don't use the SKU as your productID. productID needs to be your own unique identifier. You need to put the SKU in another field. You need to do this in order to handle multiple merchants in a single database.

    When you update your prices, you can run this:

    "select productID from crosses where sku='$data[sku]' and Merchant='Religious-Items.com'"

    This will give you the productID for the row you're updating. It's important to consider that you'll need to regularly update this information and ensure that the prices and links are always accurate. Another very important thing to consider is stock. The datafeed won't always tell you if an item has gone out of stock. Most of the time it simply won't be included in the feed. This will leave you with items in your database that are out of stock but still listed as in_stock. I have a field called Updating which is set to 1 at the start of my datafeed processing. This only happens for the merchant I'm updating at the moment. Each time I update or add a product, I set Updating to 0. After all of the rows have been processed and my data is correct I run one last query:

    "update table set Stock='0', Updating=0 where Updating = 1"

    That way all the items that weren't in the datafeed are classified as out of stock. It's very important that you maintain a solid inventory and always keep your prices up to date. Build a PHP script that will do all of this for you and run it as often as the datafeed is updated.

    Now building pages that use your database effectively is a whole other story.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  20. #20
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Thanks oscar, and Snib,

    It's working now ... Here's what they look like now (Snib, I did add auto_increment to the productID field as You said to do but it don't show up in the print view- but it's there in the regular veiw):

    You can see them here at: http://www.humblefishmall.com/ourtable.html and http://www.humblefishmall.com/ourtablebrowse.html ... See any other changes that I should make?

    Also, what variables and stuff should I be studying to extract the info from the data base tables onto php/html pages?

  21. #21
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    I wouldn't use the SAS productID field as your primary auto_increment field. I suggest that you make productID start from 1 and maintain it as your own unique number. Create a field for SKU and put the SAS productID into that field. You'll come across many datafeeds that use an SKU and you need to make sure it's not a primary key or auto_increment. You can have the same SKU in your table across multiple merchants. Your unique identifier needs to be specific to your own system. I'm just trying to make your system capable of handling more than one datafeed from places that aren't necessarily ShareASale.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  22. #22
    ABW Ambassador HumbleFish's Avatar
    Join Date
    January 18th, 2005
    Location
    Arizona
    Posts
    680
    Ok thanks Snib; but what variables and stuff should I be studying to extract the info from the data base tables onto php/html pages?

  23. #23
    Not Verif-Lidated infoTim's Avatar
    Join Date
    January 18th, 2005
    Location
    Sunny Florida
    Posts
    1,021
    You need to think about what your pages are going to look like and what information you'll need to pull out of the database in order to support that.

    - Tim
    Tim
    consultant by day, affiliate by night

  24. #24
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    You're going to need to build several PHP documents. I have 2 primary ones called category.php and product.php. I query them like this:

    /product.php?productID=XYZ
    /category.php?categoryID=ABC

    Your product.php just needs to "select * from products where productID=XYZ" Then you simply print the information from that particular row into your page. The layout and functionality is completely up to you. Category.php is much more complicated and requires a category database to contain your category structure. This can be done any number of ways, but I've found it to be the most complicated part of this entire project. You have to consider what products you want to display in a category that has no products in it directly. For example you might have Electronics > CD Players and Electronics > DVD Players but what do you show in the Electronics category before somebody clicks into either CD Players or DVD Players.

    I use a parent/child relationship in my category table and have various tools to build and edit categories. I've spent over a year on this system and it's not even to the point I'd like it to be. At least it's pretty robust and handles traffic fluidly. The hard part is making it easy enough to use and navigate.

    Good luck!
    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  25. #25
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    One important thing I forgot to mention is you need category.php in order to find the products. Otherwise how will the search engine bots or your customers find what they're looking for? You'll have to study merchant sites to see how this is laid out. It's very common. You know you could always try OS Commerce if you don't want to deal with building this huge system.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

+ Reply to Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Extract Data from a Data Feed
    By TalhaA in forum Programming / Datafeeds / Tools
    Replies: 6
    Last Post: February 25th, 2012, 05:40 PM
  2. Data Feed - Invalid Compressed Data in .gz files?
    By Bleddyn in forum Rakuten LinkShare - LS
    Replies: 7
    Last Post: November 29th, 2010, 09:30 AM
  3. Example Data Feed
    By chrissmith11 in forum Programming / Datafeeds / Tools
    Replies: 3
    Last Post: April 7th, 2008, 08:33 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •