leopoletto/robots-txt-parser
A comprehensive PHP package for parsing robots.txt files, including support for meta tags and X-Robots-Tag HTTP headers
时间:2025-11-07 21:51
nickmoline/robots-checker
Class to check a URL for robots exclusion using all possible methods of robots exclusion
时间:2016-12-26 23:21
vipnytt/useragentparser
User-Agent parser for robot rule sets
时间:2016-04-08 16:48
fourlabs/robots-bundle
Symfony2 bundle to control X-Robots-Tag HTTP header via annotations
时间:2015-08-27 16:52