A meta robots tag is an important HTML directive. The role of meta robots tag is to signal search engine crawlers how they can index and display a webpage in search results. This is placed in the <head> section of a webpage like this:
Html CopyEdit
<meta name=”robots” content=”index nofollow”>
Meta robot tag helps webmasters how to control the indexing and crawling behavior of the search engine. If a page does not have an HTML <head> section, an X-Robots-Tag (an HTTP header alternative) can be used instead.
Why Is the Meta Robots Tag Important?
The meta robots tag is useful for:
- Preventing search engines from indexing certain pages (no index).
- Stopping search engines from following links on a page (nofollow).
- Prevent snippets from catching content and more. This way, it helps to control how content appears in search results.
Common Meta Robots Tag Directives:
- noindex – Prevents a page from indexing or appearing in search results.
- Nofollow – Stops search engines from following links on the webpage.
- None – Equivalent to noindex or nofollow.
- noarchive – Prevents search engines from storing a cached version of the webpage.
- nosnippet – Disallows text snippets in search results.
- noimageindex – Stops images from being indexed in search results.
- Max-snippet – Limits the length of text snippets.
- unavailable_after – Removes a page from search results after a set date.
How to Implement a Meta Robot Tag
In, SEO plugins like Yoast and RankMath allow easy setup. Simply select the No Index option in the post settings to prevent indexing.
