SEO Configuration Of HTTP Header

HTTP Header is also known as X-Robots-Tag. Any direction explanation given in robots Meta tag is also described with X-Robots-Tag. You can see below an example of X-Robots-Tag instructing crawlers not to index a page that is depicting HTTP response results.
X-Robots-Tag: noindex 
(Use where there is no HTML like PDF file)

You can create the collection of X-Robots-Tag headers that provide multiple HTTP responses, or you can specify a comma-separated list of directives. An example of a combination of three X-Robots-Tag which has a "nofollow", "noarchive" and an "unavailable after" X-Robots-Tag. The impact of both X-Robots-Tag works differently.


X-Robots-Tag: nofollow

X-Robots-Tag: noarchive

X-Robots-Tag: unavailable after: 22 July 2018 18:30:00 PST

You can specify a user-agent before the directives. Below you can see the set of X-Robots-Tag that is showing a conditional view of the page in search results for different search engines:


X-Robots-Tag: Googlebot: nofollow

X-Robots-Tag:otherbot:noindex, nofollow

In the above examples, you can see directives those are specified without a user-agent are eligible for all crawlers means valid for all bots

Note: In SEO anchor text don’t use too much SEO "nofollow". It means you do not trust your content. Don’t add "noindex" on the page if your page is already blocked by robots.txt file. Googlebot or any other robot does not go to your page because access is denied.

seo anchor text

The above picture explains that when you want to do internal linking between the content then you can use SEO anchor text that is the tag with "href" attribute.

HTTP Header Code Sample


HTTP/1.1 200 OK

Date: Tue, 22 May 2019 12:14:43 GMT

X-Robots-Tag: googlebot: nofollow 

X-Robots-Tag: otherbot: noindex, nofollow

Note: As you see above coding, you can change your crawler means bot of any search engine. Also, you can add or remove any X-Robots-Tag.

Helpful Posts

Internal linking Questions

SEO With Hreflang Tag

Stay Happy

Like and Share.

Popular posts from this blog

SEO Vocabulary - The SEO Dictionary

Mismatched URL - Title and Description Gap

SEO Redirect: How it help us?

SEO True False Statements

What IS SEO: Optimizing The Content For Search Engine

High Quality Website

Multiple Sitemaps: How To Submit A Sitemap Index

Canonical Tag - Mobile or Desktop Version Teller

HTTP Status Code IN SEO

Log Files On Server and IN SEO

Navigate the blog from the labels. This is for your help to easily navigate the articles according to the category. Moreover, on each post, you will find the internal links that will help you to navigate the related articles.
Protected by Copyscape

Popular posts from this blog

Log Files On Server and IN SEO

What IS SEO: Optimizing The Content For Search Engine

HTTP Status Code IN SEO