spatie/crawler
Crawl all internal links found on a website
时间：2026-01-04 10:12
jaybizzle/crawler-detect
CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
时间：2026-01-04 09:58
symfony/dom-crawler
Eases DOM navigation for HTML and XML documents
时间：2026-01-04 09:51
helgesverre/firecrawl-php-sdk
PHP SDK for Firecrawl API - Web scraping and crawling service
时间：2025-10-12 06:17
iprodev/sitemap-generator-pro
A professional, production-ready PHP XML sitemap generator with advanced features: concurrency, robots.txt, caching, database storage, change detection, SEO analysis, JavaScript rendering, proxy support, webhooks, and more.
时间：2025-10-08 13:56
tfd/laravel-well-known-traffic-advice
A Laravel package that provides a standardized .well-known/traffic-advice endpoint for communicating server load, maintenance windows, and access recommendations to automated clients, crawlers, and partner services.
时间：2025-07-02 12:34
utrodus/malaysia-public-holiday
A PHP library to crawl Malaysia public holidays with filtering and caching.
时间：2025-04-19 17:00
webcrawlerapi/sdk
A PHP SDK for WebCrawler API - turn website into data
时间：2024-12-31 12:50
fkrzski/robots-txt
A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience
时间：2024-12-16 20:41
webprofil/crawler
Crawls Website and can generate backstop json
时间：2024-10-30 10:38
marcvanh/laravel-bot-block
A custom middleware package for Laravel. Temporarily blocks crawlers scanning for vulnerabilities.
时间：2024-09-06 17:25
friends-of-hyva/magento2-crawler-session
Prevent crawlers from creating a session
时间：2024-03-18 08:06