/apify-fixie-crawler

Apify Crawler for Fixie Corpus ingestion.

Primary LanguageJavaScriptMIT LicenseMIT

apify-fixie-crawler

This is an Apify Actor (using the Apify v3 SDK, based on crawlee) for crawling web content. It extracts the body, title, and other metadata from pages it crawls. In addition, it handles PDF files by downloading the raw data and saving it as base64-encoded data to the dataset.