Results 1 to 11 of 11
  1. #1
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    543
    hello All
    i just build php site (nuke)
    can googlebot crawl this link "mysite.com/modules.php?name=News&file=article&sid=4988" ?

  2. #2
    Pimp Duck popdawg's Avatar
    Join Date
    January 18th, 2005
    Location
    Take off eh?
    Posts
    3,249
    There has been lots of post here and in Webmaster World/Search Engine World about this.
    I have never had a problem with Google crawling dynamic pages on any of the three sites I set up that way, although I have used mainly asp and only recently some php.

    Something strange is afoot at Circle K

  3. #3
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    I think they have a problem with this type of thing. The SID= thing at the end may cause difficulties. There was an interview with someone from Google where they said as much - trouble is, I can't remember the details. I'll try to find the details and post them later.

    Bump this thread if I don't come back to this within a couple of days as I may have forgotten.

    Search Engine Positioning - 1 Design 4 Life

  4. #4
    Pimp Duck popdawg's Avatar
    Join Date
    January 18th, 2005
    Location
    Take off eh?
    Posts
    3,249
    Sorry, wasn't paying attention.
    Markymark is right, there have been problems with SID's, but in the context of creating a session ID for those with cookies disabled.

    Posted here:

    http://www.webmasterworld.com/forum3/5624.htm

    http://www.webmasterworld.com/forum3/5479.htm

    Shutting up now [img]/infopop/emoticons/icon_smile.gif[/img]


    Something strange is afoot at Circle K

  5. #5
    ABW Ambassador Greg Rice's Avatar
    Join Date
    January 18th, 2005
    Location
    Ohio
    Posts
    4,889
    After reading this and the numerous other posts about Google and PHP it seems Google doesn't have a problem with dynamic sites. But, from what I see in my logs, it seems like Google is avoiding all my pages that have a ? in the URL. It reads my homepage and is going through all of the directories put so far hasn't spidered any pages with a ? in them, for example:
    http://www.mysite.com/directory/prod...ProdID=xxxxxxx

    Am I just not patient enough (it's a new site) or is Google not going to spider those pages? [img]/infopop/emoticons/icon_confused.gif[/img]

  6. #6
    Pimp Duck popdawg's Avatar
    Join Date
    January 18th, 2005
    Location
    Take off eh?
    Posts
    3,249
    Well for what it's worth, a whole whack of my pages for my new computer site have been crawled and were visible as of this last update. (I thought they werent going to make it but they showed up the second day.)
    The pages in question are all .asp in this format pagename.asp?var1=123&var2=223
    My shoe store which is 100% dynamic .asp has just been crawled by google and aol as well.

    Game on!!!!

  7. #7
    Full Member
    Join Date
    January 18th, 2005
    Posts
    260
    From memory.. Googlebot being the polite little spider that it is, it doesn't like to follow links with query strings in it. This is to cut back on the enormous load it could put on a server with 1000's of pages like this. Whilst it can and does follow them, it will not dig as deep as quickly, nor visit as many pages in one hit. Mind you, I could be completely wrong on this [img]/infopop/emoticons/icon_razz.gif[/img]

  8. #8
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    TK is right, Xandman. Googlebot tries not to put too much pressure on your server when it crawls, so you are going to have to be more patient. Or you could use a mod rewrite to make the pages look static and then Googlebot will eat them up far quicker.

    Search Engine Positioning - 1 Design 4 Life

  9. #9
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    718
    Q. What's a mod rewrite?

    I've just had an almost entirely dynamic site of around 200 pages crawled and indexed by the Googlebot, and it did seem to take around 2 updates for that to happen.

    The site in questions was really slow for a few days whilst the dance was on, so maybe the bot was very busy crawling during this time?

    -----------
    JoKeThInG
    -----------

  10. #10
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    mod_rewrite is a method used to turn dynamic urls into static looking ones. Do a search at Google and you will find loads of information on it.

    Alternatively, PM me and I will give you a simpler method for making dynamic URLs look static.

    Search Engine Positioning - 1 Design 4 Life

  11. #11
    ABW Ambassador Greg Rice's Avatar
    Join Date
    January 18th, 2005
    Location
    Ohio
    Posts
    4,889
    Thanks for the reassurance Popdawg,TK and markymark. I saw sites listed on Google with similar URLs but when I saw "gangs" of 3 to 8 Googlebots enter my site and crawl around for a minute or two and then leave without touching the product specific pages, I was beginning to wonder if I did something wrong. I'm very new to PHP and considered that I may have done something that prevented the little fellas from hitting those pages.

    Thanks again for your time and consideration. [img]/infopop/emoticons/icon_biggrin.gif[/img]

  12. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Dynamic Link
    By troylemaire in forum Programming / Datafeeds / Tools
    Replies: 2
    Last Post: January 27th, 2004, 10:38 AM
  2. dynamic link ability
    By Roland in forum Commission Junction - CJ
    Replies: 0
    Last Post: October 18th, 2003, 02:40 AM
  3. New Dynamic Product Link Tool
    By Bryan Rhodes in forum Midnight Cafe'
    Replies: 17
    Last Post: November 18th, 2002, 07:06 PM
  4. Dynamic Link
    By wly in forum Commission Junction - CJ
    Replies: 1
    Last Post: November 10th, 2002, 06:20 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •