home/categories/knowledge-base/leobrival-serum-plugins-official-plugins-crawler-skills-website-crawler-skill-md
knowledge-basedocumentation
website-crawler
High-performance web crawler for discovering and mapping website structure. Use when users ask to crawl a website, map site structure, discover pages, find all URLs on a site, analyze link relationships, or generate site reports. Supports sitemap discovery, checkpoint/resume, rate limiting, and HTML report generation.
maintainer
leobrival
আপডেট হয়েছে 1/10/2026
স্টার
0
ফর্ক
0
quick start
Installation and usage
High-performance web crawler for discovering and mapping website structure. Use when users ask to crawl a website, map site structure, discover pages, find all URLs on a site, analyze link relationships, or generate site reports. Supports sitemap discovery, checkpoint/resume, rate limiting, and HTML report generation.
ইনস্টলেশন
$ install --globalskills.sh
ব্যবহার
ইনস্টল করার পর, টার্মিনালে নিচের কমান্ড চালিয়ে আপনি এই স্কিল ব্যবহার করতে পারবেন:
skills use website-crawler