搜索关键词:robots-txt,共找到 57 个结果
排序方式:
当前按 更新时间 降序 排列

arcanedev/robots

Robots.txt generator

版本:2.0.0 下载:614 Stars:7 点击:1

时间:2015-05-14 14:48

chrisguitarguy/wp-robots-txt

Manage your robots.txt file from the WordPress admin

版本:1.2 下载:376 Stars:10 点击:2

时间:2015-05-03 21:45

ellisthedev/laravel-5-robots

Robots.txt Generator for Laravel 5

版本:0.1.0 下载:214.38k Stars:34 点击:1

时间:2015-04-30 20:06

glicer/link-checker

Check broken links in html files, sitemap.xml and robots.txt

版本:未知版本 下载:13.66k Stars:28 点击:2

时间:2015-03-27 16:24

bee4/robots.txt

Robots.txt parser and matcher

版本:v2.0.3 下载:9.33k Stars:12 点击:1

时间:2015-03-13 10:06

novactive/ezseobundle

Novactive eZ SEO Bundle is an Ibexa Platform bundle for SEO simplications. metas, sitemaps, robots.txt, etc.

版本:v7.3.0 下载:246.19k Stars:26 点击:2

时间:2015-03-03 02:24

mmerian/phpcrawl

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

版本:0.8.3 下载:7.14k Stars:58 点击:2

时间:2014-12-17 09:09

t1gor/robots-txt-parser

PHP class to parse robots.txt rules according to Google, Yandex, W3C and The Web Robots Pages specifications.

版本:v0.2.5 下载:489.04k Stars:85 点击:3

时间:2014-10-29 12:08

tomverran/robots-txt-checker

Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a page

版本:1.17 下载:58.75k Stars:9 点击:2

时间:2014-04-15 20:20

healey/robots

Robots.txt Generator for Laravel 4 and 5

版本:v1.0.1 下载:202.65k Stars:40 点击:2

时间:2014-04-05 03:27

cuab/phpcrawl

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

版本:0.8.1 下载:37.99k Stars:20 点击:3

时间:2013-05-06 06:13

webignition/robots-txt-file

Models a robots.txt file

版本:3.0 下载:1.23M Stars:18 点击:2

时间:2012-07-16 21:40