How to Hide or Password Protect a Website from Google while under Development?

It’s generally a very good idea to hide your website from the search engine while it’s still in the development phase. You do not want Google or Bing to have the wrong impression that your website is of bad quality, or the content links are not stable as they come and go. This would negatively impact your search ranking, and it will take a very long time to recover from the SEO perspective.

Three Ways to Hide a Website

Let’s watch a YouTube video from John Mueller at AskGooglebot.

Summary and Comparision

We’ve made a table to compare these three methods and recap what John said.

How to Use Password Protect to Hide a Website?

We’ll take a Nignx Website on Ubuntu as an example to explain the process.

First, install the htpasswd tool, which is part of apache2-utils.

sudo apt-get update 
sudo apt-get install apache2-utils

Add user and password to a configuration file. The flag -c means creating a new file .htpasswd. So if you would like to add more users, you’ll need to ignore -c for the second user.

sudo htpasswd -c /etc/nginx/.htpasswd tom
sudo htpasswd /etc/nginx/.htpasswd jerry

After this, you can view the .htpasswd file you just created.

sudo more /etc/nginx/.htpasswd
tom:$apr1$liMw.lEn$uLo2/ZCJ4HYbnCEyvU/Cr/
jerry:$apr1$uoM9YQwL$6R/1ovVV4MLPxKDtEvzo91

In the Nginx configuration file for your site, add the following lines to the server block, no matter it listens to 80 for HTTP or listen to 443 for HTTPS.

server {
    ...
    auth_basic "Restricted";
    auth_basic_user_file /etc/nginx/.htpasswd;
    ...
}

For more details about auth_basic, you may refer to the official Nginx document ‘Restricting Access with HTTP Basic Authentication‘.

At last, you’ll need to restart your Nginx server.

sudo systemctl restart nginx

In your web browser, visit your website again, and you’ll get the password prompt.

I have noticed one thing. If your site is HTTP, the string defined by the auth_basic directive is not prompted on the dialog. If your site is HTTP, which of course I don’t recommend, the auth_basic directive shows on the dialog. But either way, you’ll need the auth_basic directive, otherwise, password protection does not work at all. The official Nginx ‘auth_basic‘ document does not give too much clue. If you know the reason, please leave a comment below.

Final Thoughts on Hiding a Website

I highly recommend hiding the entire site. You can gradually roll out your pages when they are ready. Never rush as you do not want to mess up with search engines. If your project is a little complicated and cannot use WordPress or Drupal to set up the site in a matter of hours, the chance of Google or Bing indexing the incomplete site is high. Although sometimes, when you really would like search engines to index, it seems to take forever.

After hiding your site, you can start to think about other elements of your site, such as URL strategy. I recommend you continue to read ‘Best Practices to Make WordPress Blog URL SEO Friendly‘.

Let us know your thoughts by adding a comment below.