HTML::RobotsMETA is a simple HTML::Parser subclass that extracts robots exclusion information from meta tags. Author: Daisuke Maki WWW: http://search.cpan.org/dist/HTML-RobotsMETA/