productivity-toolstools
web-crawler
High-performance Rust web crawler with stealth mode, LLM-ready Markdown export, multi-format output, sitemap discovery, and robots.txt support. Optimized for content extraction, site mapping, structure analysis, and LLM/RAG pipelines.
maintainer
leobrival
์
๋ฐ์ดํธ๋จ 1/10/2026
์คํ
0
ํฌํฌ
0
quick start
Installation and usage
High-performance Rust web crawler with stealth mode, LLM-ready Markdown export, multi-format output, sitemap discovery, and robots.txt support. Optimized for content extraction, site mapping, structure analysis, and LLM/RAG pipelines.
์ค์น
$ install --globalskills.sh
์ฌ์ฉ๋ฒ
์ค์น ํ ํฐ๋ฏธ๋์์ ๋ค์ ๋ช ๋ น์ ์คํํ์ฌ ์ด ์คํฌ์ ์ฌ์ฉํ ์ ์์ต๋๋ค:
skills use web-crawler