SEO Configuration Of HTTP Header

HTTP Header is also known as X-Robots-Tag. Any direction explanation given in robots Meta tag is also described with X-Robots-Tag. You can see below an example of X-Robots-Tag instructing crawlers not to index a page that is depicting HTTP response results.
X-Robots-Tag: noindex 
(Use where there is no HTML like PDF file)

You can create the collection of X-Robots-Tag headers that provide multiple HTTP responses, or you can specify a comma-separated list of directives. An example of a combination of three X-Robots-Tag which has a "nofollow", "noarchive" and an "unavailable after" X-Robots-Tag. The impact of both X-Robots-Tag works differently.


X-Robots-Tag: nofollow

X-Robots-Tag: noarchive

X-Robots-Tag: unavailable after: 22 July 2018 18:30:00 PST

You can specify a user-agent before the directives. Below you can see the set of X-Robots-Tag that is showing a conditional view of the page in search results for different search engines:


X-Robots-Tag: Googlebot: nofollow

X-Robots-Tag:otherbot:noindex, nofollow

In the above examples, you can see directives those are specified without a user-agent are eligible for all crawlers means valid for all bots

Note: In SEO anchor text don’t use too much SEO "nofollow". It means you do not trust your content. Don’t add "noindex" on the page if your page is already blocked by robots.txt file. Googlebot or any other robot does not go to your page because access is denied.

seo anchor text

The above picture explains that when you want to do internal linking between the content then you can use SEO anchor text that is the tag with "href" attribute.

HTTP Header Code Sample


HTTP/1.1 200 OK

Date: Tue, 22 May 2019 12:14:43 GMT

X-Robots-Tag: googlebot: nofollow 

X-Robots-Tag: otherbot: noindex, nofollow

Note: As you see above coding, you can change your crawler means bot of any search engine. Also, you can add or remove any X-Robots-Tag.

Helpful Posts

Internal linking Questions

SEO With Hreflang Tag

Stay Happy

Like and Share.

Popular posts from this blog

SEO Queries Searched By Bloggers

SEO Vocabulary - Basic Words Of SEO

Blogger SEO: How To Do?

Multiple Sitemaps: How To Submit A Sitemap Index

Must See Helpful Blogger Question - Answer

What IS SEO: Optimizing The Content For Search Engine

SEO Marketing - What it is and How To Do?

Digital Marketing SEO - What it includes?

SEO Link Juice - The Genuine Link Hub

SEMrush Company and Tool Information

Navigate the blog from the labels. This is for your help to easily navigate the articles according to the category. Moreover, on each post, you will find the internal links that will help you to navigate the related articles.
Protected by Copyscape

Popular posts from this blog

SEO Vocabulary - Basic Words Of SEO

What IS SEO: Optimizing The Content For Search Engine

Blogger SEO: How To Do?