indexing crawling

How to stop crawling or indexing of a website on search engines.

Do you want to know how you can prevent indexing your WordPress website or a specific page of a website and remove it from the search?

Then, you are at the right place.

I made this article short and simple to understand this topic quickly without any jargon. 

So, let’s start

There may be a lot of reasons why you may want to stop indexing your website.

Here, I have mentioned some grounds on which someone prevents the website from search engine crawlers or bots. 

The reason for doing it.

  1. You just created a website, but the designing portion of the website is still pending.
  2. This may be required when you want to hide a website during the time for maintenance of the website or a page.
  3. When you create a landing page for a specific purpose- selling a product, signing up for a course, thank you page, etc.

Now, I will tell you guys,

The steps to hide the complete website.

  1. Log in to your WordPress Dashboard.
  2. Click on the “Settings” option from the left side panel and click on the “Reading” option.
  3. Scroll down on the right side and you will find the “Discourage search engines from indexing this site” option and click on the box, and it checkmarks it.
  4. And the last thing to do is apply the settings on the website. Just click the “save changes” option.

Note: After this step followed. 

A code is added in the robot.txt file that notifies the crawlers not to index the website.

But there may be a chance of getting crawlers later on.

To fully disable the website from indexing, you have to protect the website by creating a password.

You can create a security password for the website by installing a plugin by the name of the “Password Protected” plugin.

The steps to hide a Specific Webpage.

There’s no other option in WordPress, particularly for a specific page.

But you can install a plugin for it and, it will solve the purpose.

I use RankMath for SEO purposes and, it has the option to disable indexing for a specific page.

For RankMath Plugin, the steps are:

  1. Go to the “Pages” tab and click on the “Edit” button for that page you want to hide from the search engine robots.
  2. After that, on the right panel from the Rank Math panel, go to the “Advanced” tab (suitcase icon).
  3. Under the “Robots Meta” tab in the panel, click on the “No Index” checkbox.
  4. And now, click on the “Publish/Update” button. That’s all.

Note: These are the two methods which I have mentioned. You can use to hide a page or website from indexing that is WordPress setting and, the other one is by using a plugin.

But, if you have external links on the page or the website and the links by default are set to “follow“. Then the search engine will still follow the webpage.

And if you don’t want that, you can set it to the “no follow” option available in the panel. 

If you want to know more tips and tricks and other related aspects, then click here and sign up (It’s completely free). 

After you sign up, you will get emails related to a new post uploaded on this website.

Also, share your thoughts and suggestions on which topic you want a blog in the comments section or click here. And I will try to create a blog on the topic you asked for.