best Robot.txt File

A robots.txt file is a text file which plays a vital role in search engine optimization. It helps the Google bots to crawl your WordPress blog. The entire mechanism of robots.txt file can be designed to allow or disallow certain links of your WordPress blog. Every new bloggers have confusion which is the best method to create a robots.txt file. So I have listed the easiest and best way to create this file. You just have to follow my instructions and you’re done with creating the best robots.txt file, without a robots.txt file Google cannot crawl your WordPress blog. So it is essential that you create this file with proper care and instructions.

Best Robots.txt File

Here I have added the best robots.txt file which you can use this file making fewer changes and uploading it to your WordPress blog directory.

User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /recommended/
Disallow: /comments/feed/
Disallow: /trackback/
Disallow: /index.php
Disallow: /xmlrpc.php
Disallow: /wp-content/plugins/

User-agent: NinjaBot
Allow: /

User-agent: Mediapartners-Google*
Allow: /

User-agent: Googlebot-Image
Allow: /wp-content/uploads/

User-agent: Adsbot-Google
Allow: /

User-agent: Googlebot-Mobile
Allow: /


Things Needed To Create Robots.txt File

You will need your notepad to create robots.txt file.
You will need the robots contents to fill in your newly created file.

How to create best Robots.txt file


Create your new text document which is nothing but .txt file.

When you create a new .txt file. The file will be blank.

Now you to rename the file to robots and save it.

Now you have to add all the contents of robot.txt file that I have provided. You just need to copy paste the contents of best robot.txt file which I have given to you. You just need to change the web address because the robot.txt file contains my blog page URL. You just need to replace my blog page URL with your URL and save the text file that you created as robots.txt.

Now you have successfully created your new robots file which you can upload to your WordPress blog.

How to add robot.txt to your WordPress blog


The new robot text file that you created can be added to your root of your WordPress blog using the file manager or there are certain WordPress plug-ins which you can install which has the option to upload robots.txt file.

How to test new robot.txt


Create Best WordPress Robots.txt File


Now that everything that we have created and added is successful. It is now the testing time of our newly created robots file. You have to go to Google Webmaster tools or Google search console a new name of Webmaster tools for testing your newly uploaded file.

I have simplified this article to help you in much easier way to add robot.txt.

I hope this article is has helped you to know more about robots which crawl your webpages and index it. If you liked this article please do subscribe and leave your comments.



Tags –¬†Create Best WordPress Robots.txt File –¬†How to Make Robots