ROBOTStxt

    Home | Download | PayPal Donate Bookmark and Share
 

ROBOTStxt is an editor for robots.txt. Robots.txt files are used to direct bots which pages to fetch from a website. More information regarding robots.txt can be found on the wikipedia article.

Features:
  • Edit Robots.txt
  • View Robots.txt
  • Open Robots.txt from WWW
How To:

This application is a very simple application to edit or view robots.txt files. You can either open an existing robots.txt file (either from a remote location on the www, or from your local disk). You can then start adding entries. The most important entries to add are the User-agent which identifies a user agent (use * to match all) and any Disallow paths. To delete an entry simply right click on it and click on delete. A sample robots.txt file can be found over here. Please note that robots.txt file should reside in the root directory of your website and should be named robots.txt for the bot to find it.

 
Main Window:
ROBOTStxt Main Window
   

 Download Now       v1.0  

Requires Microsoft .NET Framework 2.0

 
 
 
Please Donate for ROBOTStxt. Thanks to all those who donated.
Requirements:
  • Windows 2000, Windows XP, Windows 2003, Windows Vista, Windows 2008, Windows 7.
  • Microsoft .NET Framework 2.0 Download
 
Any ideas, comments or feature requests contact me on email ian@iannet.org

Ian     www.iannet.org
Copyright © iannet 2010