t1gor/robots-txt-parser
PHP class to parse robots.txt rules according to Google, Yandex, W3C and The Web Robots Pages specifications.
时间：2014-10-29 12:08
tomverran/robots-txt-checker
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a page
时间：2014-04-15 20:20
healey/robots
Robots.txt Generator for Laravel 4 and 5
时间：2014-04-05 03:27
nobrainerweb/silverstripe-robots-noindex
Show a warning in the CMS when site is set to noindex
时间：2013-12-11 10:27
cuab/phpcrawl
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
时间：2013-05-06 06:13
tractorcow/silverstripe-robots
Simple robots generation module for Silverstripe
时间：2013-02-17 21:57
webignition/robots-txt-file
Models a robots.txt file
时间：2012-07-16 21:40