Results 1 to 10 of 10
  1. #1
    Yup, Sure ... now let me check ... Cagles Mill's Avatar
    Join Date
    January 18th, 2005
    Location
    The Midwest
    Posts
    646
    I'm new to using datafeeds and just completed building my first Perl based feed. I've searched through many of the datafeed threads and yet although I think the answer has already been posted more than once, it has not been posted in terms I understand.

    If the pages are dynamically generated, then the search engines will not pick them up?! So.. you have to convert the pages you want in the search engines into static pages, which can then in turn still point to other dynamic links? Or, you can leave the potential pages as dynamically generated as long as you use a URL redirect to get rid of all the '+', '&' and other such sysmbols in the URL?

    What I want to do is to get not just the main page into the search engine, but also each of the main product categories.

    Does this all make sense?

    "The crowd has no hands." - Soren Kierkegaard

  2. #2
    Member
    Join Date
    January 18th, 2005
    Posts
    53
    search engines will pick up dynamic pages. just look at the urls in google, theres a ton of them with abc.php?ddd=xxx&fff=9898 or abc.asp?keyword=blah&aaa=blahblah format. The main reson to mask the param's is to get some keywords in the url. Some believe that it help's there position. I've done it both ways and it hard to tell if one way's better than the other, I have some high ranking pages both ways.


    if you want to change them instead of using a redirect read up on apache's mod-rewrite

    Charlie

    "Changes in latitudes, changes in attitudes nothing remains quite the same.
    With all of our running and all of our cunning, If we couldn't laugh, we would
    all go insane." -- Jimmy Buffett

  3. #3
    Member
    Join Date
    January 18th, 2005
    Posts
    99
    The problem with search engines is not that the contents are dynamically generated. How would they know? The contents are compiled before it's ever served to the visitor.

    One parameter in the URL is generally indexed easily by Google. The more parameters in the URL, the more time it takes for them to index.

  4. #4
    Just Lurking
    Join Date
    January 18th, 2005
    Posts
    1,263
    Changing your program structure my might help with these dynamic datafeeds.

    Instead of using a front controller where everything goes through say index.php. Try using a filter type structure. Where every page includes an initializer that processes any parameters it may find and a finalizer to clean up afterwards. These two routines can be easily included by Apache or PHP using prepend/append. This structure allows any file to processing any parameters. You can name your files whatever you want. This structure makes your pages a lot easier to understand too.

    Using this structure your page in PHP would look something like this:

    <pre class="ip-ubbcode-code-pre">&lt;?php
    include_once 'initializer.php';
    include $psg['path_templates'].'tpl_header.php';
    include $psg['path_templates'].'tpl_left.php';
    ?&gt;

    HTML CONTENT HERE&lt;br&gt;
    or call database for what you need

    &lt;?php
    include $psg['path_templates'].'tpl_right.php';
    include $psg['path_templates'].'tpl_footer.php';
    include 'finalizer.php';
    ?&gt;</pre>

    The initializer can take care of loading common function, initializing the variables and session, processing any commands or parameters. The finalizer takes care of any logging or anything else you need to do at the end of every page.

    I'm a little tired so if this makes no sense...sorry. I will say I've found this structure (or the fancy term Pattern) much more flexiable and easier to maintain.

    ------------------------------
    "If all the newbies are going to start writing ebooks maybe I should get started on mine!" -- Buddha

  5. #5
    Intergalactic Trader IGshop's Avatar
    Join Date
    January 18th, 2005
    Posts
    87
    I was also under the impression that google would not deeply crawl dynamic links but that changed yesterday when google spent the day deep-crawling not one but two of my datafeed sites. (However, I have seen yahoo do this before.) These sites go down two levels: category, then product. I also have two sites where the dynamic links have been converted to "nice" links which have not been deep crawled by google. Google's been poking around them so I think it'll be going in soon.

    Jim

  6. #6
    ABW Ambassador CrazyGuy's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,463
    I think alot of this misunderstanding comes from the fact that many PHP-driven dynamic site use the built-in PHP session ID to move visitors around a site.

    Those are unique on every visit, so the SE spiders are understandably wary of them ...

    So, dynamic pages do get indexed. All other things being equal, do they get ranked as highly? Opinions vary, but it seems logical to keep your dynamic URLs as simple as possible - leave empty parameters out rather than having lots of empty values, that kind of thing.

    ________________________________________________
    "CrazyGuy your more than crazy...your pathetic."

    Mike & Charlie ...

  7. #7
    Member
    Join Date
    January 18th, 2005
    Posts
    53
    Crazy, forgot about those session ids. they can be a problem for spiders. I don't use session id's since I don't run have a shopping cart on my sites. I do know that osCommerce actually has code that will kill/drop the session id out of the url if the visitor is a spider.

    Charlie

    "Changes in latitudes, changes in attitudes nothing remains quite the same.
    With all of our running and all of our cunning, If we couldn't laugh, we would
    all go insane." -- Jimmy Buffett

  8. #8
    Yup, Sure ... now let me check ... Cagles Mill's Avatar
    Join Date
    January 18th, 2005
    Location
    The Midwest
    Posts
    646
    This is great! I didn't expect so many replies so quickly.

    So let me see if I have this straight. Let's say I have a datafeed from widgets.com and they have 10 categories of products, with 50 products in each of the 10 categories.

    So... I set up a page www.mysite.com/widgets.html. This is a static page of course, but let's say I put in a search form and links to the perl script for 9 of the 10 categories (let's say I don't want to promote one category). If a person clicks one one of those 9 links, they will get the first page for all 50 items in that category. If I have it set up to display 10 items per page, there will be 5 pages within each category. I realize the search engines won't make any use of the search form I mentioned including.

    OK, assuming all the above was clear enough, the initial widgets.html page will of course be found by the search engines as usual. It will perhaps take longer for the search engines to find the first page displayed for each of the categories, but because I have links pointing to each category, those pages will eventually be found even though they are dynamically generated. Then, last but not least, the search engines will eventually find pages 2,3,4, and 5 within each category as well, even though there is no static link pointing to them?! Meanwhile, since I ignored one of the ten categories, the search engines will never find any pages for that particular category.

    Have I got it straight???

    "The crowd has no hands." - Soren Kierkegaard

  9. #9
    Member
    Join Date
    January 18th, 2005
    Posts
    53
    I think your on the right track but I think your mixed up about dynamic pages and static links. To the visitor (spider or human) any link on a page no matter if it's xyz.html or xyz.php?categoryid=1 is a link. A link is a link. how the page getscreated is dymanic(built on the fly) or static(pre built).

    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR> the search engines will eventually find pages 2,3,4, and 5 within each category as well, even though there is no static link pointing to them <HR></BLOCKQUOTE>

    yes if there is a link on page one that goes to page 2 and so on the link will be followed by the spider, just like a human would follow it.

    remember a spider has to way to know how the page is created or does it care it just read's the output and follows the links

    Charlie

    "Changes in latitudes, changes in attitudes nothing remains quite the same.
    With all of our running and all of our cunning, If we couldn't laugh, we would
    all go insane." -- Jimmy Buffett

  10. #10
    Yup, Sure ... now let me check ... Cagles Mill's Avatar
    Join Date
    January 18th, 2005
    Location
    The Midwest
    Posts
    646
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>I think your on the right track but I think your mixed up about dynamic pages and static links. <HR></BLOCKQUOTE>

    You're right, that's been my problem.

    But now I think I've got it and about all I can say is WOW! No wonder datafeed users seem to like them so well. You can both add content and large numbers of pages fast. On the other hand, it's sort of scary. It would be easy to not only become guilty of spamming the search engines, but if there is only one or two products that actually sells well, you could spam your own page out of a good ranking. Dangerous stuff here!

    I hate to quote a comic book, but as Spiderman would say, "With great power comes great responsibility" (or something along those lines).

    A big thanks to everbody for your responses.

    "The crowd has no hands." - Soren Kierkegaard

  11. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Generate static pages for Clickbank
    By Millionaire in forum ClickBank
    Replies: 11
    Last Post: March 31st, 2005, 05:42 PM
  2. Update static pages?
    By cbgaloot in forum Programming / Datafeeds / Tools
    Replies: 6
    Last Post: February 24th, 2005, 02:31 PM
  3. From dynamic pages to static - good approach?
    By eggerda in forum Search Engine Optimization
    Replies: 0
    Last Post: October 1st, 2003, 04:55 PM
  4. php static pages
    By awstewar in forum Programming / Datafeeds / Tools
    Replies: 1
    Last Post: July 15th, 2003, 12:10 PM
  5. Do SE's Like Static or Dynamic Pages Best?
    By midnightcoder in forum Search Engine Optimization
    Replies: 2
    Last Post: June 17th, 2003, 01:54 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •