Results 1 to 17 of 17
  1. #1
    Full Member dak142's Avatar
    Join Date
    January 18th, 2005
    Posts
    386
    Google spidered over 16,000 pages on my site this month. However, when I check the pages indexed it only shows about 900 or so. Will all the spidered pages eventually show up in the index?

  2. #2
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    I'd like to know this too. I guess if there is nothing wrong with the pages they should add them. I've noticed it takes a while before they get added sometimes.

  3. #3
    Newbie
    Join Date
    January 18th, 2005
    Posts
    6
    We have seen a similar issue with our new email marketing site - Google spidered the entire site which isn't too big but we are not seeing all the pages.

    One thing a buddy of mine suggested that we may need to have a site page with all of the pages. Not sure if this is the solution.

    vweb
    Email Marketing Choice
    www.emailmarketingchoice.com

  4. #4
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    Your buddy is referring to what we call a sitemap. It's not necessary if you have good linking already. It's great if you have a site that's hard to link together but most sites don't need one.

  5. #5
    Newbie
    Join Date
    January 18th, 2005
    Posts
    10
    If the domain is fairly new than it can take 3-6 months before anything ranks. If you have a bit older domain with lots of backlinks than new pages rank fairly quickly. Think about it, 16000 pages on a new domain - not very natural

  6. #6
    ABW Ambassador Doug247's Avatar
    Join Date
    January 18th, 2005
    Location
    DE USA
    Posts
    931
    Ok here is the deal... My friend knows this guy who was told by someone who is really familiar with the way google indexes pages. If a page was crawled on Monday thru Thursday it takes a month to get into the index. Friday crawls usually take 2 months to get indexed while pages crawled on Sat and Sun can take 3 - 6 month to get indexed. This only applies to .html pages. If a page ends in .htm it may take twice as long to get indexed. Another factor google considers is weather a site was scanned in the AM or PM. If it was scanned in the AM Mon th Thur and the page is .html there is no change in the time it will take to get indexed. However if a site is crawled in the PM you can add a day on the length of time it will take to get indexed. This is because by 12PM on any given day the entire USA is awake and using the internet. This tends to slow things down somewhat. One more thing google looks at is if you still have matress tags on your matresses...Be aware that if you have submitted a site to google and have removed the tags from any matresses in you house google will rapidly index your entire site but it will place no higher then 1,457,390 thus giving you very little traffic. And for those of you who don't already know....All this is a Bunch of BS and I haven't a clue about how google rates, ranks, or indexes sites.

    But seriously 3-6 Mos is a good ballpark number.
    Thanks,
    Doug

  7. #7
    ABW Ambassador mousejockey's Avatar
    Join Date
    January 18th, 2005
    Posts
    2,494
    I put up a new site on Friday,new domain, no datafeeds, no incoming links, only outgoing links,small 10 page site and I'm surprised to find google spidered it yesterday.

    Couldn't believe google came to visit so quickly, I'll keep an eye on how long it takes for google traffic to arrive.

  8. #8
    Prince of Content Vinny O'Hare's Avatar
    Join Date
    January 18th, 2005
    Posts
    3,126
    Doug the funny thing about your post is if you didnt put in the last sentence in about a week it would be on all the forums as a FACT lol
    Vinny O'Hare - OPM - Contact Info email: vinny at teamloxly.com ~ 702-582-6742 Twitter

  9. #9
    ABW Veteran Student Heyder's Avatar
    Join Date
    January 18th, 2005
    Posts
    5,482
    I bet it would too. Basically to answer any and all google related questions remember these words "I don't know!".

  10. #10
    Full Member dak142's Avatar
    Join Date
    January 18th, 2005
    Posts
    386
    Funny thing happened, when I started this thread I had alomst 900 indexed pages. I was just about to go to bed and took a look at my logs and noticed a HUGE traffic spike from Google. My site now has almost 3000 pages indexed. Just for others to estimate by, google hit me hard between 6th - 11th of last month, and the beginning of this month.

    That said, it looks like it took 6 weeks for for the first hit to get picked up. I will contine to update this thread and track the trend as more happens.

  11. #11
    Newbie
    Join Date
    January 18th, 2005
    Posts
    6
    David:

    Thanks for the update. I wanted to know if your PageRank of Google improved/moved from the base=0?

    Our site pages are indexed but for some reason we still see PageRank=0. Not sure why this is happenening.

    vweb
    Email Marketing Choice: Find, Compare and Choose Email Marketing Solutions

  12. #12
    Full Member
    Join Date
    January 18th, 2005
    Posts
    461
    3000, 6000, 16,000 indexed pages. I have to wonder how many of those pages are from data feeds, how many are there for fooling the serps and/or spamming them, how many are useless to the searcher, how many are duplicates, etc etc etc...

    I got a feeling that many of you are a reason why affiliate marketing is a target for the search engines with the end result being the beginning of the end of affiliate marketing all together through search engines unless there is a way to filter some of this BS out. I have to believe there is a lot of irresponsibility around here.

    At the very least, some laziness for sure. Affiliate marketing is being destroyed by the likes of some of you. That seems very clear the more I read posts in this forum. The funny thing is, half of you don't have a clue! It's all about fooling the search engines with some of you. You call yourselves affiliate marketers. What a joke!

  13. #13
    ABW Ambassador Doug247's Avatar
    Join Date
    January 18th, 2005
    Location
    DE USA
    Posts
    931
    Dirk,

    What is you definition of Affiliate marketing??
    Thanks,
    Doug

  14. #14
    ABW Adviser Panel Dynamoo's Avatar
    Join Date
    January 18th, 2005
    Location
    Opposite the Slough of Despond
    Posts
    5,465
    Great post about the spidering, Doug.

    Here's my take on Google's indexing based on my observations.

    Put simply, Google re-indexes popular sites more often than less popular ones. And by popular, I'm talking about clickthroughs counted from the Google SERPs.

    So, for a new site with little traffic it can take weeks to get indexed. For a popular site, it can take just a couple of days (or even less for a *really* popular site).

    But here's a thing.. PageRank doesn't make a whole lot of difference. I have a low-traffic PR6 site that takes up to two weeks to get an update. I have a high-traffic PR5 site that takes about two days.

    GoogleBot *always* seems to be around regardless. But if the site isn't popular, he seems to put the pages on the "to do" list to get around to later. Google also has a behaviour with new sites where it will start with just the index page and then add more pages progressively.
    Innovative advertising with Slimeware Corporation and Telephore. Mail-order fuel with Petrol Direct.

  15. #15
    2005 Linkshare Golden Link Award Winner  ecomcity's Avatar
    Join Date
    January 18th, 2005
    Location
    St Clair Shores MI.
    Posts
    17,328
    Doug his definition includes a person who takes time to do a better job at promoting certain merchant's products better then they do themselves. The more value he adds to that niche' within a merchants ecatalog the more money he earns. The SERPs come naturally and do not disappear.

    Not to hard to find examples here of someone with a datafeed, who launches a merchant site inside a weekend with a bare bones feed template. They like to call themselves merchants, but they're just frontend posers. The hook is to pass out the same feed to all takers for spamming the SERPs to launch their program.

    His warning is the SERP spammers, e-mail spammers, browser spammers have so pissed off the duped shoppers and so cluttered the SERPs with SEM tricks their is no room for legit value-added resellers seeking long term commissions. I sure hope the SE spammers, networks Adwhore AM's, feel the wrath of shoppers seeking to win back the power of their mouse.
    Webmaster's... Mike and Charlie

    "What have you done today to put real value into a referral click...from a shoppers viewpoint!"

  16. #16
    Newbie johnmcowen's Avatar
    Join Date
    January 18th, 2005
    Location
    Massachusetts
    Posts
    33
    Hi guys,

    I have seen the same thing happening recently when I launched a new site. GGbot came and "ate" the site over 350 pages. That was 2 weeks ago. Two days later, 40 pages show up, and I started to get searches from GG...a week later, nothing anymore, still 40 pages indexed, the same ones...I did some research, and found an interesting article about the Google "sandbox" effect, check this out:

    "The trend seems to be this: when new sites are first uploaded they receive close Google attention and decent rankings for their keywords. After two to three weeks these new sites drop in the rankings and pages from these sites drop from the index as well.

    The poster who first noticed this phenomenon tested his findings by placing pages from his new site into an older site. These new pages soon ranked well for their keywords on the older site. (Please read the post for full details, including other.)

    Logan, a JimWorld moderator, said, "new sites seem to react different to changes (slower) than established sites when it comes to rankings - based on my experiences with new sites since florida/austin/brandy..."

    It's curious that, as jimish reported in JimWorld, for his site, "Pagerank has been going up steadily, but what good is that if you don't show up in the results."

    Logan also offered a potential explanation, "my guess is that there is an initial 'local rank' established based on the contents indexed from the site. But afterwards, there seems to be some delay with the role links play."
    "

    From webproworld.com/viewtopic.php?t=18075

    JM

  17. #17
    ABW Adviser Panel Dynamoo's Avatar
    Join Date
    January 18th, 2005
    Location
    Opposite the Slough of Despond
    Posts
    5,465
    Yes, I've certainly noticed new sites being listed and then dropping out for a couple of weeks.

    The last new site I did took sooo long to get indexed and the updates were so slow that eventually I pulled the content back into a more established site where it does much better.
    Innovative advertising with Slimeware Corporation and Telephore. Mail-order fuel with Petrol Direct.

  18. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Google Can Now Index . . . Flash!
    By webdesignhq in forum Search Engine Optimization
    Replies: 7
    Last Post: December 30th, 2004, 11:37 AM
  2. Google Index Double!
    By ahmar in forum Search Engine Optimization
    Replies: 11
    Last Post: November 16th, 2004, 09:21 AM
  3. geez when is google going to index?
    By jc101 in forum Search Engine Optimization
    Replies: 1
    Last Post: May 16th, 2003, 04:53 AM
  4. New Google Index
    By cubbbb in forum Search Engine Optimization
    Replies: 4
    Last Post: April 11th, 2002, 11:09 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •