[Arfa]

I have a Wordpress based site, which aggregates lots of RSS feeds and then lets the user search them (as well as merging in search results from elsewhere). Essentially I'm using the Wordpress post database as an index for my own custom search engine. Once the content passes a certain age, I ditch it. For visitors this is a good specialised search engine and a nice aggregated feed of latest posts.

Of course, aggregated and duplicate content is seriously going to pull down my Google index. I am bolstering the site with good original content to back things up. But I'm guessing its going to be an up hill struggle to work against the aggregated content I'm using.

I was thinking, can I tell spiders to not index the aggregated content and only search the rest, with rel=nofollow, robots.txt etc. Is this likely to remedy the situation?
Are there any other strategies I should pursue?

[Convergence]

Originally Posted by Arfa

I was thinking, can I tell spiders to not index the aggregated content and only search the rest, with rel=nofollow, robots.txt etc. Is this likely to remedy the situation?

That would be very helpful. If the bots can't index it - it really doesn't exist.

the Google also obeys googleon/googleoff tags. Search for those if you're not familiar.

Also, if you can modify your header meta tags on those pages put in noarchive tags. The Google still archives pages without the noarchive tag, even if you have noindex tags or block them in robots.txt.

Cheeky bastards, eh!?!...