productivity-toolstools
web-crawler
High-performance Rust web crawler with stealth mode, LLM-ready Markdown export, multi-format output, sitemap discovery, and robots.txt support. Optimized for content extraction, site mapping, structure analysis, and LLM/RAG pipelines.
maintainer
leobrival
اپ ڈیٹ ہوا 1/10/2026
اسٹارز
0
فورکس
0
quick start
Installation and usage
High-performance Rust web crawler with stealth mode, LLM-ready Markdown export, multi-format output, sitemap discovery, and robots.txt support. Optimized for content extraction, site mapping, structure analysis, and LLM/RAG pipelines.
انسٹالیشن
$ install --globalskills.sh
استعمال
انسٹال کرنے کے بعد، آپ یہ اسکل ٹرمینل میں درج ذیل کمانڈ چلا کر استعمال کر سکتے ہیں:
skills use web-crawler