Results 1 to 18 of 18
  1. #1
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    Adding urls to google sitemap question
    Still haven't really determined yet how useful these are. One site has them, other site doesn't. Both have sitemap pages visitors actually see that I've made for them. One site's a whole lot smaller tho so don't know if that's a factor.

    Regarding the google sitemap I had made for free from a site online:

    It lists the urls on my site from that day it was made. Since then, there's been several more added. Those urls are not in alphabetical order.

    First question is: Is it ok for me to manually type in urls to pages that have since been added to my site by putting in the same info as the other urls have, just with their different urls??

    And if so, can I move those urls and the info that's included ( the sitemap gobblygook that I don't know what it means, but can tell it's the same for each url ) with them so that all the urls are in alphabetical order. It would be soooooo much easier for me to keep track of em that way when I have to add them.

    Or, will that confuse the poor little Google site map guy and it'll take him longer to figure out where the pages for my site are or make any difference in a negative way at all I guess??

  2. #2
    notary sojac Herb ԿԬ's Avatar
    Join Date
    January 18th, 2005
    Location
    Central/Western NY State
    Posts
    7,741
    yes

    yes
    and
    no, I don't think google spider needs any special order. they don't hit my sites and go in order very often.

  3. #3
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    Thank you very much Herb ԿԬ for answering all of my questions.

  4. #4
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    Did you try the GsiteCrawler? It's a freebie that works very well and has pretty active Google group. I use it on all my sites and it will even help you spot duplicate content.


  5. #5
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    1,744
    Thanks for the link bumpaw - I think I will give this one a try.

  6. #6
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    Nope, I wasn't aware of that... thank you very much bumpaw

  7. #7
    Affiliate Manager Afilyit's Avatar
    Join Date
    April 25th, 2008
    Location
    Staten Island, NY
    Posts
    348
    Quote Originally Posted by bumpaw
    Did you try the GsiteCrawler? It's a freebie that works very well and has pretty active Google group. I use it on all my sites and it will even help you spot duplicate content.
    I have tried using gsitecrawler. It just takes too long for me (many hours).
    Does anyone have knowledge on google sitemap.xml? Is it really necessary?
    Can it "hurt" a web site? After all, wouldn't google be able to crawl your pages without it?

  8. #8
    ABW Ambassador purplebear's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,960
    There seems to be all kinds of opinions here about it.

    Definitely am no expert. but don't think it's necessary given the fact that a bunch here don't submit them.

    I'm sure I probably haven't read all of the comments but don't recall anybody sayin by doing so could actually hurt a site.

    Think their opposition to it is more based on just not lettin Google know more than they already know about their sites. It's possible there may be some negatives said about it, may turn up some if you make a search for it?

    I just have been experimenting with one site having one and the other not. I do like bein able to have access to some of the info Google has from them under the webmaster tools. Not sure if the others have helpful info or not. Have submitted to them also. hmmm maybe I should be checking to see if they do have anything.

  9. #9
    ABW Ambassador Boom or Bust's Avatar
    Join Date
    February 3rd, 2008
    Posts
    3,955
    Quote Originally Posted by Afilyit
    I have tried using gsitecrawler. It just takes too long for me (many hours).
    Does anyone have knowledge on google sitemap.xml? Is it really necessary?
    Can it "hurt" a web site? After all, wouldn't google be able to crawl your pages without it?
    I'm a firm believer in submitting sitemaps to Google because it works very well for me. Site updates and additions are indexed usually within 24 hours. Since I'm right here with the servers mine compiles at LAN speeds but I can imagine how long it would take otherwise especially if you like to sit and watch it. Otherwise, let it run all night if necessary.



    X

  10. #10
    Affiliate Manager Afilyit's Avatar
    Join Date
    April 25th, 2008
    Location
    Staten Island, NY
    Posts
    348
    I guess this is my main concern

    Using the sitemap generator, I get about 6000 links. A lot of it
    is nonsense. Are all those URLS really necessary?

    And the real concern, if a submitted sitemap is NOT complete, does that
    in effect tell google to stop crawling from there?

  11. #11
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    And the real concern, if a submitted sitemap is NOT complete, does that
    in effect tell google to stop crawling from there?
    A sitemap is not something Google will obey one way or the other. They will still crawl all your links, but the sitemap is designed for you to indicate your priorities.

    The biggest site I ever used GsiteCrawler on was around 75,000 urls and it did it in about 3 hours. I do find if your site has a lot of problem links it will take longer, but that is something you want to get away from. There are good filters in there if you take time to set it up.

    Edit: I just did 386 urls in 4 mins.


  12. #12
    ABW Ambassador Boom or Bust's Avatar
    Join Date
    February 3rd, 2008
    Posts
    3,955
    Quote Originally Posted by Afilyit
    Using the sitemap generator, I get about 6000 links. A lot of it
    is nonsense. Are all those URLS really necessary?
    I have a lot of pages that I filter out of the sitemap in the generator. It's easy using wildcards or regular expressions. Compile time would be lower if you filtered out all the unwanted pages.



    X

  13. #13
    ABW Ambassador 2busy's Avatar
    Join Date
    January 17th, 2005
    Location
    Tropical Mountaintop
    Posts
    5,636
    I use GSiteCrawler too, with filters it crawls my sites in minutes. Without filters it took hours and hours. I don't want it to list every dynamic page on the sitemap, Google finds them anyway. The pagenames for some dynamic pages are generated by the merchants' product name so I have no way to know what is current, temporary, duplicate, etc. Many times I have regretted starting using sitemaps because changing or adding pages means an update but just as many times I am happy that I have the sitemaps because it gets new stuff indexed superfast. Even if you don't upload your sitemap in GWT it is a very useful tool for your own use to help weed out cruft/stuff you forgot you put up there.

  14. #14
    Affiliate Manager Afilyit's Avatar
    Join Date
    April 25th, 2008
    Location
    Staten Island, NY
    Posts
    348
    Thanks for the help.

    Another question - what to submit

    http://yourdomain.com/
    or
    http://www.yourdomain.com/

    I read somewhere (I believe at Google) that both are NOT the same,
    yet they both lead to the same place. Does it really make a difference?
    Thanks

  15. #15
    ABW Ambassador writerguy's Avatar
    Join Date
    January 17th, 2005
    Location
    Springfield, Missouri, USA
    Posts
    3,248
    Quote Originally Posted by Afilyit
    Thanks for the help.

    Another question - what to submit

    http://yourdomain.com/
    or
    http://www.yourdomain.com/

    I read somewhere (I believe at Google) that both are NOT the same,
    yet they both lead to the same place. Does it really make a difference?
    Thanks
    I always use www.yourdomain.com instead of yourdomain.com because I personally believe people are just too "trained" to put "www." into every URL address from long habit. But I have no research to back that up.

    I also seem to recall there's something somewhere in Google's webmaster tools that allows you to tell Google to treat yourdomain.com and www.yourdomain.com as the same thing when they index the site.

    Anybody know more specifics about that? Or am I just delusional again? (Hazards of aging.)
    Generate more fake news.

  16. #16
    Comfortably Numb John Powell's Avatar
    Join Date
    October 17th, 2005
    Location
    Bayou Country, LA
    Posts
    3,432
    I read somewhere (I believe at Google) that both are NOT the same,
    yet they both lead to the same place. Does it really make a difference?
    If you have different URLs that point to the same page they treat that as duplicate content and see that as a negative. The consensus seems to be that you should pick one and be consistent in your internal links.


  17. #17
    ABW Ambassador 2busy's Avatar
    Join Date
    January 17th, 2005
    Location
    Tropical Mountaintop
    Posts
    5,636
    There are many ways to write the URL that opens up the main page of your site, but you need to select one URL and use it only. In case you have links from other sites that are not using that one URL you should add the snippet to your .htaccess file that will redirect those other URLs to your selected URL. That information is available here at ABW in other threads.
    There is plenty of information available from Google on how to avoid problems, how to set up your .htaccess and more. Some helpful area are:

    http://www.google.com/webmasters/tour/tour1.html

    http://www.google.com/webmasters/

    http://googlewebmastercentral.blogspot.com/

    http://groups.google.com/group/Google_Webmaster_Help

  18. #18
    ABW Ambassador Boom or Bust's Avatar
    Join Date
    February 3rd, 2008
    Posts
    3,955
    Quote Originally Posted by Afilyit
    Thanks for the help.

    Another question - what to submit

    http://yourdomain.com/
    or
    http://www.yourdomain.com/

    I read somewhere (I believe at Google) that both are NOT the same,
    yet they both lead to the same place. Does it really make a difference?
    Thanks
    I just searched one of my own domain names with/without www. at the beginning. Got two results with www. and many results without. BUT...
    Will explicit searches be made for yourdomain.com or wwwyourdomain.com? I don't think so. People normally search using keywords, not domain names. So I really don't think it makes a difference one way or the other.



    X

  19. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Sitemap Question
    By klm1159 in forum Search Engine Optimization
    Replies: 8
    Last Post: September 6th, 2011, 11:47 AM
  2. sitemap.pl Google Sitemap Generator
    By cusimano in forum Cusimano.com Scripts
    Replies: 16
    Last Post: April 23rd, 2010, 07:07 PM
  3. Adding Anchors to GC urls
    By Borat42 in forum GoldenCAN
    Replies: 0
    Last Post: March 12th, 2009, 01:44 PM
  4. Betty and Google Sitemap Verification
    By John Powell in forum Cusimano.com Scripts
    Replies: 1
    Last Post: April 29th, 2006, 08:59 PM
  5. Google sitemap changes
    By infoTim in forum Search Engine Optimization
    Replies: 6
    Last Post: November 21st, 2005, 02:33 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •