Robots Meta tag

This Robots Meta tag give the message to search engines, need to index the current page in search engine or not.

Robots Meta tag – Syntax

<meta name=”robots” content=”index/noindex, follow/nofollow”>

Instead of ‘robots’ we can use search engine’s bots (user-agents) names, like Googlebot,Yahoo Slurp, bingbot, etc….

Robots Meta tag – Examples

Meaning for Each Robots Meta tag follows

<meta name=”robots” content=”index, follow”>
  • The spider(user-agent/bot) will now index your web page.
  • The spider will not only index the first webpage of your website but also all your other webpages.
<meta name=”robots” content=”noindex, nofollow”>
  • The spider(user-agent/bot) will not index this page and will not crawl through the rest of your webpages.
<meta name=”robots” content=”index, nofollow”>
  • The spider will now index only this page and stops there(not crawl rest of the pages).
<meta name=”robots” content=”noindex, follow”>
  • The spider will not index this page but will crawl through the rest of the pages on your website.
<meta name=”Googlebot” content=”noindex, follow”>
  • The google spider alone will not index this page but will crawl through the rest of the pages on your website.
  • Like this for all spiders we can separately give robots meta tag.

Leave a Reply

Your email address will not be published. Required fields are marked *