Block Content Access For Search Engine

Blocking content access in Google and other search engines is not easy to understand who are beginners. Retrieving web pages on your site by the crawler is monitor through the robots.txt file.

Allow: /URL is used when you have one directory is blocked but you one of URL to access by search engine crawler.

But Blocking URL comes under the Disallow: /

Disallow: /search/label/SEO

In the above code label, SEO is blocked for indexing and crawling.

My robots.txt file

User-agent: Mediapartners-Google 

Disallow: User-agent: * 

Disallow: /search Allow: / 

Sitemap: https://www.mssalotra.com/sitemap.xml

Read more about robots.txt file with Google article.

You can block URL with NoIndex also. But the crawler will visit on URL and then understand the NoIndex. Finally, your URL will not come in search.

You can also say "NoSnippet" to search crawler and with this, you can block your content access in SERP.

For any help ask a question.

Stay Happy

Like and Share

Popular posts from this blog

How To Use Redirect On Blogger?

Google's SEO Tools

What is Cloaking in SEO?

Mismatched URL - Title and Description Gap

SEO Redirect: How it help us?

Black Hat SEO - The Negative Task

SEO Vocabulary - Basic Words Of SEO

What IS SEO: Optimizing The Content For Search Engine

Website's Success

Blogger Comments - How To set?

Navigate the blog from the labels. This is for your help to easily navigate the articles according to the category. Moreover, on each post, you will find the internal links that will help you to navigate the related articles.
Protected by Copyscape

Popular posts from this blog

Log Files On Server IN SEO

What IS SEO: Optimizing The Content For Search Engine

SEO Vocabulary - Basic Words Of SEO