loveduckie/silverstripe-robots
A Silverstripe module for managing the robots.txt file from the CMS.
时间：2024-11-17 11:08
bopoda/robots-txt-parser
PHP Class for parsing robots.txt files according to Google, Yandex specifications.
时间：2019-01-02 12:29
vipnytt/robotstxtparser
Robots.txt parsing library, with full support for every directive and specification.
时间：2016-04-08 10:58
fourlabs/robots-bundle
Symfony2 bundle to control X-Robots-Tag HTTP header via annotations
时间：2015-08-27 16:52
t1gor/robots-txt-parser
PHP class to parse robots.txt rules according to Google, Yandex, W3C and The Web Robots Pages specifications.
时间：2014-10-29 12:08