1 2 3 4
Robots is a simple Ruby library to parse robots.txt files. WWW: https://github.com/fizx/robots WWW: https://github.com/fizx/robots