Filewatcher File Search File Search
Catalog
Content Search
» » » » » »

perl-WWW-RobotRules

Database of robots.txt-derived permissions

This module parses /robots.txt files as specified in "A Standard for Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html>. Webmasters can use the /robots.txt file to forbid conforming robots from accessing parts of their web site.
Homepage:
Package version:6.01
Architecture:src
Distribution:Fedora
Filename:perl-WWW-RobotRules-6.01-4.fc17.src.rpm

Browse inside perl-WWW-RobotRules-6.01-4.fc17.src.rpm

868 B2014-11-29HEADER  view  485+ mirrors
2.34 KB2014-11-29perl-WWW-RobotRules.spec  view
8.83 KB2014-11-29WWW-RobotRules-6.01.tar.gz  view  1845+ mirrors

Download perl-WWW-RobotRules-6.01-4.fc17.src.rpm

Results 1 - 1 of 1
Help - FTP Sites List - Software Dir.
Search over 15 billion files
© 1997-2017 FileWatcher.com