Results 1 to 10 of 10
  1. #1
    Member
    Join Date
    January 18th, 2005
    Posts
    94
    Getting all your pages Spidered
    Hi,

    I have been doing alot of reading here about datafeeds and am thinking of trying them out.

    My one concern is: How do you get all 3000+ pages spidered?

    Does it depend on your PR?

    thanks

    mopek

  2. #2
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    We were just having this discussion over here:

    http://forum.abestweb.com/showthread.php?t=59674

    Might want to check it out. To answer your question, you need some very strong PR to get datafeed pages indexed. I have a site that's going on 8 years old with a solid PR and Google won't even fully index the few datafeed pages I have. It's pretty difficult to get datafeed pages into search engines lately.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  3. #3
    Crazy Cat Lady Heidi's Avatar
    Join Date
    January 18th, 2005
    Location
    Rochester, NY
    Posts
    1,685
    Good navigation & site maps helps alot. The biggest mistake I see people make is that their navigation to their inner pages sucks.
    Heidi
    "Happy are those who dream dreams and are willing to pay the price to make them come true"

  4. #4
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    It will SPIDER them with a lot less than PR8. About PR4~PR5 will do ya for 3000 pages. (Remember that it doesn't know what kind of pages they are, until it checks!)

    But, like Scott alludes to, just getting *spidered* isn't enough. Getting a visible rank is an art unto itself...and one that generates much debate.
    There is no knowledge that is not power. ~Hemingway

  5. #5
    Member
    Join Date
    January 18th, 2005
    Posts
    94
    Thanks Scott Heidi & Leader!

    I have been reading about the debate you speak of Leader via Scott's link (above). Great discussion!

    So basically, you need at good PR5 first to get them indexed and then you need to make it unique so it can get ranked.

    Heidi - can you give me an example of good vs bad navigation to inner pages?

    Also, I am finding it harder and harder to get PR since google keeps lower it across the board every so often. I don't care about PR so much for placement or status or traffic. I care about it for getting my pages spidered.

    mopek

  6. #6
    Full Member Tech Evangelist's Avatar
    Join Date
    March 16th, 2005
    Location
    Mesa, AZ
    Posts
    374
    Google and other search engines generally will not spider a page that takes more than three clicks to reach. Keep your site design shallow and use a site map with a link to it on every page.
    There's good, fast and cheap. Pick any two.
    [url=http://www.topranksolutions.com]Phoenix SEO[/url] :: [url=http://www.tech-evangelist.com/category/affiliate-marketing/]Affiliate Marketing Tutorials[/url]

  7. #7
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    you need at good PR5 first
    Well, actually I meant, "a bit more than four but not necessarily all the way to 5." I've got lots of PR4 pages in. With the toolbar, though, I don't know whether that's a "low 4" or a "high 4." It just says "4"...

    I do notice that with a 5, indexing seems to happen quicker, but as long as I've got at least 4 it seems to get 'em. Things are a lot more sporadic at 3. Also, there's a NEW SITE effect where sites that don't have any PR yet may be spidered well, but that seems to work better with smaller sites.

    Also, pay attention to what Heidi and TechEvangalist said about the navigation. Even the highest PR'd site can have pages buried beyond where robots will bother to go, if they're not careful.

    IMO 3000 pages isn't a lot, and PR4 isn't too hard to get either. Trying to get Googlebot to eat 100,000 pages, now THAT'S a challenge! For a really huge site (100,000+ pages), a PR8 may indeed help. I've gotten it to eat about 28,000 pages, and which ones are in at any particular time varies.
    There is no knowledge that is not power. ~Hemingway

  8. #8
    Super Sh!t Stirrer SSanf's Avatar
    Join Date
    January 18th, 2005
    Posts
    9,944
    If you have a link to a site map or two from your home page and the site maps contain links to all the pages, wouldn't any page be just two clicks away?
    Comments are opinion unless otherwise noted. Remember, pillage first. Then burn. Half of all people in the world have IQs under 100. You best learn to trust ol' SSanf!

  9. #9
    ABW Ambassador
    Join Date
    January 18th, 2005
    Posts
    804
    Maybe not....if you've got 5k pages, they aren't going to go on 1 or 2 pages of a site map....you'll probably need multiples broken down into sections (maybe alphabetical).

    Ken

  10. #10
    Member
    Join Date
    January 18th, 2005
    Posts
    94
    Excellent!

    Thanks for the info everyone.


  11. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Do option values is forms get spidered?
    By loxly in forum Search Engine Optimization
    Replies: 2
    Last Post: April 28th, 2004, 02:52 PM
  2. IrishPhotoshop site not spidered
    By Radegast in forum ShareASale - SAS
    Replies: 6
    Last Post: October 13th, 2003, 04:40 AM
  3. Anyone have any tips to get spidered more often?
    By nute3 in forum Search Engine Optimization
    Replies: 9
    Last Post: July 20th, 2002, 11:27 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •