Pinned Repositories
actor-whitepaper
This whitepaper describes a new concept for building serverless microapps called Actors, which are easy to develop, share, integrate, and build upon. Actors are a reincarnation of the UNIX philosophy for programs running in the cloud.
apify-cli
Apify command-line interface helps you create, develop, build and run Apify actors, and manage the Apify cloud platform.
apify-mcp-server
Apify MCP server (tools for web scraping, data extraction, and automation)
crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
fingerprint-suite
Browser fingerprinting tools for anonymizing your scrapers. Developed by Apify.
got-scraping
HTTP client made for scraping based on got.
impit
impit | rust library for browser impersonation
mcp-server-rag-web-browser
A MCP Server for the RAG Web Browser Actor
proxy-chain
Node.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining.
apify's Repositories
apify/browser-pool
A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.
apify/fingerprint-generator
Generates realistic browser fingerprints
apify/header-generator
NodeJs package for generating browser-like headers.
apify/fingerprint-injector
Home of fingerprint injector.
apify/covid-19
Open APIs with statistics about Covid-19
apify/actor-content-checker
You can use this act to monitor any page's content and get a notification when content changes.
apify/apify-ts
Crawlee dev repo
apify/devtools-server
Runs a simple server that allows you to connect to Chrome DevTools running on dynamic hosts, not only localhost.
apify/actor-quick-start
Contains a boilerplate of an Apify actor to help you get started quickly build your own actors.
apify/actor-scrapy-executor
Apify actor to run web spiders written in Python in the Scrapy library
apify/act-crawler-results-to-s3
Apify actor to upload crawler results to AWS S3.
apify/browser-headers-generator
Package generating randomized browser-like headers.
apify/actor-example-proxy-intercept-request
Example: Intercept requests from https connection using "Man in the middle" proxy solution.
apify/actor-algolia-website-indexer
Apify actor that crawls website and indexes selected web pages to Algolia index. It's used to power the search on https://help.apify.com
apify/actor-example-php
Example of Apify actor using PHP
apify/apify-php-tutorial
apify/http-request
A HTTP request library for Node.js, with a common-sense API, support for Brotli compression and without bugs in "request" NPM package
apify/scraping-tools-js
A library of utility functions that make scraping, data extraction and usage of headless browsers easier and faster.
apify/actor-crawler-cheerio
DEPRECATED: An actor that crawls websites and parses HTML pages using Cheerio library. Supports recursive crawling as well as URL lists.
apify/actor-crawler-puppeteer
DEPRECATED: An Apify actor that enables crawling of websites using headless Chrome and Puppeteer. The actor is highly customizable and supports recursive crawling of websites as well as lists of URLs.
apify/apify-sdk-v2
Snapshot of Apify SDK v2 + sdk.apify.com website. This project is no longer maintained. See the https://github.com/apify/apify-sdk-js repo instead!
apify/apify-tsconfig
TypeScript configuration shared across projects in Apify.
apify/generative-bayesian-network
apify/nodelocaldns
Temporary location of the nodelocaldns chart until it is upstreamed to helm/charts
apify/apify-jsdoc-template
apify/crawlee-migration-example
apify/generator-networks-creator
apify/ow-cjs
apify/test-travis-github-migration
apify/workshop-cape-town
Resources and final project code of a Developer Summit workshop in Cape Town.