Subscribe to
Posts
Comments

Search engine guys are thinking to reduce rank or ban certain links of a website which has their site search results crawled by them be it Google, MSN, Yahoo, Ask etc.,

You can check in the search engine by using site:pagerank0.com (replace with your site name) and see whether the user search results are crawled.

Search results crawled

To remove them from getting crawled in your robots.txt file just add:

Disallow: /?s=

In my case it was ?s, if your site is search, then you have to replace s with search. Additionaly if you are using google webmaster tool, you can submit that url to remove it from their index.

Leave a Reply