Results 1 to 7 of 7
  1. #1
    Newbie
    Join Date
    January 18th, 2005
    Posts
    9
    I'm wondering if anyone here has successfully used a portion of the ODP data dump in conjunction with WebMerge?

    I myself am having problems with the download. Most who use ODP (dmoz.org) data parse it into a database and serve it dynamically.

    I, however, simply wish to take a portion (sub-category)of the directory and make static pages.

    Seems "do-able", just wondering if anyone has tried and succeeded. Of course, you'd have to play with the raw data in Excel for a while first.

    My other problem is that I'm trying to download the whole content portion and, even though it is noted as being 490 MB or so, it keeps downloading well into the 500MB range, after which I stopped the download. For all I know, it could be 400 GB. It would be nice if downloads were broken down into top-level categories, but I suppose that's too much to ask.

    At any rate, now that I've rambled on...anyone have any experience here?

    ~Bleddyn

  2. #2
    You are in, or you are out ... choose!
    Join Date
    January 18th, 2005
    Posts
    459
    You might be better off using DMOZ Extractor to extract the portion of DMOZ you want into MSAccess, then export into MSExcel, then Webmerge into your pages. Much better than downloading the whole dump to use only a small part, and the Extractor results would probably be in an easier format to use.

    Woz

    dWoz - serious webmaster resources.

  3. #3
    Newbie
    Join Date
    January 18th, 2005
    Posts
    9
    Thanks Woz - I'm going to give that a go tomorrow...

  4. #4
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    Not that I don't want you to use WebMerge, but DMOZ extractor will export the data to Links 2, which is free for non-commercial use. If you want to manipulate the data or let people add/update there own links it is an awesome combination.

    You can see it in action here:
    http://collectibles.cool-networks.net/collecting

    If you just want static data, then WebMerge is the way to go.

    Deborah Carney
    Link up!
    Submit your sites, if you fit the criteria
    http://coolcraftsites.com/directory
    http://coolpetsites.com/petlinks

  5. #5
    Affiliate Manager
    Join Date
    January 18th, 2005
    Location
    Los Angeles, California
    Posts
    1,913
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by loxly:
    Not that I don't want you to use WebMerge, but DMOZ extractor will export the data to Links 2, which is free for non-commercial use.<HR></BLOCKQUOTE>

    But if I read their license agreement correctly it's $150 for business use.

    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>If you want to manipulate the data or let people add/update there own links it is an awesome combination.

    You can see it in action here:
    http://collectibles.cool-networks.net/collecting

    If you just want static data, then WebMerge is the way to go.<HR></BLOCKQUOTE>

    It looks like a fine tool, and it has me wondering: if you're building sites that don't require real-time processing of user input, is there any other aspect of Links you'd like to see in WebMerge?

    --
    Richard Gaskin
    Fourth World Media Corporation
    Developer of WebMerge: Publish any database on any site
    ___________________________________________________________
    Ambassador@FourthWorld.com http://www.FourthWorld.com
    Tel: 323-225-3717 AIM: FourthWorldInc

  6. #6
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    Yes, for commercial use their is a licence fee. This is the flat file version. I basically use it to create directories from DMOZ dumps and then let the link owners take over. It is great for sites that want to have a resource directory and allow users to submit updates. It sends me an email to let me know there are new additions or modifications and then I go in once a day or so and approve or deny the updates, then build the pages. Like I mentioned in my other post, the Links program is for sites that need the ability to be updated with user submitted information fairly frequently. After a newsletter mailing I may update the pages several times in one day. And although I have a couple of sites running 15,000+ links, Links 2 is only recommended to have 4000-5000 links for the most efficient use. There is also a built in newsletter and email manager. There is an add on for template management that I have installed on my copy of the program.

    Data management is all online through an admin interface. So additions, deletions and modifications are easy to do, and the pages get rebuilt online and quickly.

    There is a "Big Brother" that is $450 that uses MySql and has lots more customizability, LinksSql which you can see in action at http://coolpetsites.com/petlinks One of the big features of this program is to "Build Changed Pages" only. So I don't have to rebuild the entire directory each time changes are submitted. This version has user created accounts, email and newsletter management, and template management.

    WebMerge and the 2 Links programs fill different needs, in my opinion, since the Links programs are designed for online use and high site visitor interaction. I haven't looked into using WebMerge to create a site from the links data files yet, although I may shortly. I want to make several specialty sites and don't want to pay licensing fees for each. For a static directory WebMerge should be able to read the data files created by the DMOZ dump program to create the pages. There are separate files for categories and for the links information, so creating a static directory would be pretty simple.

    I don't know if any of this is helpful, you can email me (dc @ loxly.com) for more detailed information if you still have questions.

    I'm not a Links reseller or affiliate or anything, I just like using the programs for directories Others use Links for affiliate program datafeeds and links, I don't like it for that alone. I see WebMerge being more useful in that arena. I see the two programs being used together to create some awesome websites in the near future, once I recover the data from a dying 250 gig external hd that has all my feeds and links backups...

    Deborah Carney
    Link up!
    Submit your sites, if you fit the criteria
    http://coolcraftsites.com/directory
    http://coolpetsites.com/petlinks

  7. #7
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    532
    Bleddyn,

    Another option is to have WebMerge spit out .shtml pages instead of .html pages (if your server supports parsed pages) and then use a program that parses the data on the DMOZ server by keyword, caches that data on your server for speed, and then writes the data to your pages via SSI. This option allows you to actively update the pages as content changes on DMOZ and yet if using SSI the links are written to the pages just like they were hand coded. I have a site where I incorporate DMOZ data in just this fashion. Just another option.

    Affiliate-Blog.com

  8. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Detailed data feed and WebMerge instructions
    By buy_online in forum WebMerge (Fourthworld.com)
    Replies: 41
    Last Post: April 30th, 2004, 02:41 PM
  2. IF you use WebMerge what tool do you currently use to clean up your data feed?
    By buy_online in forum WebMerge (Fourthworld.com)
    Replies: 23
    Last Post: February 16th, 2004, 01:11 PM
  3. Anyone managed to use webmerge with XML data feeds?
    By womanht in forum WebMerge (Fourthworld.com)
    Replies: 7
    Last Post: April 29th, 2003, 04:09 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •