Apify Website Content Crawler
Load data from Apify Website Content Crawler.
Last updated
Was this helpful?
Load data from Apify Website Content Crawler.
Last updated
Was this helpful?
is a web scraping and data extraction platform that provides an app store with more than a thousand ready-made cloud tools called Actors.
The Actor can deeply crawl websites, clean their HTML by removing a cookies modals, footers, or navigation, and then transform the HTML into Markdown. This Markdown can then be stored in a vector database for semantic search or Retrieval-Augmented Generation (RAG).
Input one or more URLs (separated by commas) where the crawler will start, e.g https://innovativesol.com/.
(Optional) Specify additional parameters such as maximum crawling depth and the maximum number of pages to crawl.
Loads website content as a Document.
(Optional) Connect .
Connect Apify API (create a new credential with your ).
Select the crawler type. Refer to .
