So I've been watching Yahoo! Slurp carefully lately and I've noticed it's trying to retrieve pages on my site that were deleted several years ago with no incoming links remaining. Worst part is this is *all* Yahoo is grabbing.

When I look over my results on Yahoo I see the same page over and over and over again. They must have over 1000 copies of the same page indexed with slightly different URL's because the sessionID my forum creates is random. I'm going to need to add some code to make sure bots don't get a session ID. I hope this cleans up some of these results.

I can't believe their database could be so sloppy, they've got too much useless information and not enough new information. I tried to find pages that I created within the last several months to no avail. I think Yahoo is probably full of junk results so they don't want to grab anything else.

Does anybody have any suggestions or ideas about why this is happening? I'm also open to suggestions on getting my newer pages indexed. Yahoo! Slurp hits my site regularly, every few minutes or so, but it's just wasting it's time grabbing the same documents over and over again.

- Scott