Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Generate a robots.txt file for free with this easy-to-use tool. Make sure search engine crawlers are crawling & indexing your site.

A file called robots.txt can be placed in the root folder of your website to help search engines index it more accurately. Website crawlers, or robots, are used by search engines like Google to review all of the material on your website. You may not want certain areas of your website crawled and included in user search results, such as the admin page. These pages can be expressly ignored by adding them to the file. The Robots Exclusion Protocol is used in Robots.txt files. With the inputs of pages to be omitted, this website will easily build the file for you.