/rbtxt

rbtxt is a CLI tool for quick access to robots.txt

Primary LanguageGoMIT LicenseMIT

README

rbtxt is a tool that allows you to check the robots.txt of each site from the CLI.

Feature

  • Check existence of robots.txt
  • Display the contents of robots.txt
  • Save the content of robots.txt to local file

Get standard

This program requires Go to be installed. If not, please install Go 1.21.0 or higher from the official website.

To install the latest version of rbtxt, execute the following command.

go install github.com/aqyuki/rbtxt@latest

For advanced users

You can install any version of rbtxt by specifying the tag.

go install github.com/aqyuki/rbtxt@<tag>

For Contributor

All contributions are welcome. All contributors may participate in this project.

  • Bug report
  • Bug fix
  • Propose new functions and functional improvements

License

This program is released under the MIT License.