Results 1 to 10 of 10
  1. #1
    ABW Ambassador Doug247's Avatar
    Join Date
    January 18th, 2005
    Location
    DE USA
    Posts
    931
    Daily Crawl vs Your Site Updates
    Everyone always says is good SEO practice to update or add content to a site on a daily basis, which I understand. My question is why? My main landing page is crawled only about once a month. If I spend 15 - 30 minutes a day updating the changes are only every seen by the google bot once a month.

    I have a site with displays 10 random products on the main landing page on every load. Could that be seen as an 'update' by google? I also try to frequently ad content to the main landing page. The content is in the form of polls, news blurbs, etc. I still have not seen the Google bot or any other SE bot visiting the main page more than once or twice a month.

    Is there a way to request a daily crawl so that your updates and content additions are indexed?
    Thanks,
    Doug

  2. #2
    Troll Killer and best Snooper!
    I decide when the pigs fly!
    Rhea's Avatar
    Join Date
    January 18th, 2005
    Location
    New York, USA
    Posts
    6,195
    The more frequently you update your site the more frequently it will be crawled. The logic is that there's no need to crawl a site daily that isn't updated daily.

  3. #3
    ABW Ambassador 2busy's Avatar
    Join Date
    January 17th, 2005
    Location
    Tropical Mountaintop
    Posts
    5,636
    In your Google Webmaster Tools account you can change the crawl rate to increase or decrease the frequency.

  4. #4
    ABW Ambassador Doug247's Avatar
    Join Date
    January 18th, 2005
    Location
    DE USA
    Posts
    931
    Quote Originally Posted by 2busy
    In your Google Webmaster Tools account you can change the crawl rate to increase or decrease the frequency.
    I saw that but I could not increase the frequency.
    Thanks,
    Doug

  5. #5
    http and a telephoto
    Join Date
    January 18th, 2005
    Location
    NYC
    Posts
    17,708
    Quote Originally Posted by itsupportnotes
    I saw that but I could not increase the frequency.
    They won't always let you. Like Rhea said, when they start seeing more frequent updates to the site they start stopping by more often.
    Deborah Carney
    TeamLoxly.com BookGoodies.com ABCsPlus.com

  6. #6
    ABW Ambassador Boom or Bust's Avatar
    Join Date
    February 3rd, 2008
    Posts
    3,955
    Uploading a new sitemap initiates a crawl and it usually happens within a day or two. But of course that increases the workload of updating because you have to generate a new sitemap every time.



    X

  7. #7
    ABW Ambassador 2busy's Avatar
    Join Date
    January 17th, 2005
    Location
    Tropical Mountaintop
    Posts
    5,636
    because you have to generate a new sitemap every time
    Or put up the Cusimano script that generates a new sitemap dynamically

  8. #8
    Moderator
    Join Date
    April 6th, 2006
    Posts
    2,689
    You can't change the default crawl-rate of google in webmaster tools, just slow it down (I think increase only works if you have previously decreased it).

    When I launched my first site, it was daily updates from the get-go on the front page.. as the site moved up in SERP, so did the crawl. This was old-fashioned html, and new pages were added at the rate of 1-2/wk. Which meant the menu expanded on the front page (to include the new pages)

    A CMS-based expansion site on the same domain now has 2-3 new posts daily.

    My last home page crawl was Oct 5th.

    I'm not saying this is the right way to do things, just wanted to share my experience. I spend a good chunk of time tweaking/updating... it has served well for ranking which gives me time to work on conversions.

    Perhaps a better way (for me) would have been to focus on conversions before SEO, but I was very new to affiliate marketing 2 yrs ago. So I worked on what I knew...

  9. #9
    Moderator BurgerBoy's Avatar
    Join Date
    January 18th, 2005
    Location
    jacked by sylon www.sylonddos.weebly.com
    Posts
    9,618
    I only update my sites when a datafeed is updated. Something that can be very seldom - but all of my sites get crawled every day by G.

    Go figure.

    Most of them are several years old though. Maybe the age of the site helps the crawl rate?

    Vietnam Veteran 1966-1970 USASA
    ABW Forum Rules - Advertise At ABW

  10. #10
    Newbie
    Join Date
    October 6th, 2008
    Location
    Canada
    Posts
    1
    Quote Originally Posted by itsupportnotes
    Everyone always says is good SEO practice to update or add content to a site on a daily basis, which I understand. My question is why? My main landing page is crawled only about once a month. If I spend 15 - 30 minutes a day updating the changes are only every seen by the google bot once a month.

    I have a site with displays 10 random products on the main landing page on every load. Could that be seen as an 'update' by google? I also try to frequently ad content to the main landing page. The content is in the form of polls, news blurbs, etc. I still have not seen the Google bot or any other SE bot visiting the main page more than once or twice a month.

    Is there a way to request a daily crawl so that your updates and content additions are indexed?
    If your new contents can help you to gain some new keywords for your webpages, then I would say you want Google to visit your new contents when you update those contents on your website. Other than that I don't think it is useful to get Google to visit your website because your website will not gain anything.

  11. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Daily Crawl
    By Doug247 in forum Midnight Cafe'
    Replies: 4
    Last Post: September 19th, 2008, 03:50 PM
  2. BettyMills Datafeed Daily Updates
    By Victor Hanna in forum BettyMills
    Replies: 8
    Last Post: June 12th, 2007, 12:29 PM
  3. JEGEM.com datafeed now updates daily
    By jegem in forum Programming / Datafeeds / Tools
    Replies: 2
    Last Post: February 15th, 2004, 04:09 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •