HyperCrawl

Super-fast web crawling for LLM development

151 votes·12 comments·May 26, 2024

About

This is a zero-latency web crawler especially designed for retrieval-based LLM development