Home
Tools

Generate Online SEO-friendly robots.txt For Blogger

Generate a professional SEO-friendly robots.txt for your Blogger blog. Follow simple steps to create and add your robots.txt file for better indexing.
Author
Professional SEO robots.txt Generator - Tools
Table of Contents

How to generate a Professional SEO robots.txt?

To easily create XML sitemaps for your Blogger blogs for improved SEO, simply follow the four steps listed below.

  1. In the given textbox, type your domain name without the http:// or https:// prefix.
  2. Click the XML sitemap generator button labelled as "Generate robot.txt".
  3. Additionally, our service will create your sitemap instantaneously.
  4. Copy the Generated SEO robots.txt

NOTE: Please input your blogger website URL in www.blogname.tld without http:// or https://. (eg. www.exonoob.in)


Reset Generate robots.txt
Copy robots.txt

How to add an XML robots.txt sitemap in Blogger blog?

Here are the steps to add an XML robots.txt sitemap to your Google Blogger blog :

Step #1 :- Sign into your Blogger Dashboard and Select your blog website.

Step #2 :- In the left sidebar, navigate to "Settings".

Step #3 :- In the Settings page, go to "Privacy" category and enable "Visible to search engines"

Step #4 :- Scroll all the way down to the "Crawlers and indexing" category.

Step #5 :- You will find the "Enable custom robots.txt" option toogle it enable, for any reason, it is disabled.

Step #6 :- Now Click on the "Custom robots.txt" and code here.

Step #7 :- Select "Save", if you wish to save the changes that have been made.

Step #8 :- If you have not done this, head over to Google Search Console, formerly known as Webmaster Tools and add your blog.

Step #9 :- Within Search Console, go to the item termed "Sitemaps" within the "Indexing" segment.

Step #10 :- But again input the same sitemap URL as the one inputted in Blogger then submit it.

You sitemap will be submitted to Google and it may take a few days for Google to start processing your sitemap.

This process could help make Google to crawl your blog easier and possibly increase your search engine ranking.

What is robots.txt?

Think of robots.txt as a set of instructions for search engine bots. It tells them which parts of your site they can access and which parts they should ignore. This file sits in your blog's root directory and acts like a traffic cop for web crawlers.

Why is robots.txt important for Blogger users?

  • Control what gets indexed: You can prevent certain pages or sections from appearing in search results.
  • Manage crawl rate: Help search engines crawl your site more efficiently.
  • Protect sensitive content: Keep private areas of your blog hidden from public view.

Basic robots.txt commands:

  • User-agent: Specifies which bots the rules apply to
  • Allow: Permits access to specific pages or directories
  • Disallow: Blocks access to specific pages or directories

This tells all bots they can access your entire site except for the search results pages.

User-agent: *
Disallow: /search
Allow: /

Advanced robots.txt techniques:

Block specific bots:

User-agent: BadBot
Disallow: /

Prevent indexing of duplicate content:

User-agent: *
Disallow: /p/

Block access to certain file types:

User-agent: *
Disallow: /*.pdf$

Post a Comment