scraping-websites
There are 1772 repositories under scraping-websites topic.
MontFerret/ferret
Declarative web scraping
Anorov/cloudflare-scrape
A Python module to bypass Cloudflare's anti-bot page.
elixir-crawly/crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
AmmeySaini/Edu-Mail-Generator
Generate Free Edu Mail(s) within minutes
Python-World/Python_and_the_Web
Build Bots, Scrape a website or use an API to solve a problem.
csbun/thal
译文：Puppeteer 与 Chrome Headless —— 从入门到爬虫
slotix/dataflowkit
Extract structured data from web sites. Web sites scraping.
KTZgraph/sarenka
OSINT tool - gets data from services like shodan, censys etc. in one app
gildas-lormeau/single-file-cli
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
josephlimtech/linkedin-profile-scraper-api
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON.
avidLearnerInProgress/python-automation-scripts
Simple yet powerful automation stuffs.
spekulatius/PHPScraper
A universal web-util for PHP.
baptisteArno/tinking
🧶 Extract data from any website without code, just clicks.
unixfox/pupflare
A webpage proxy that request through Chromium (puppeteer) - can be used to bypass Cloudflare anti bot / anti ddos on any application (like curl)
lkuffo/web-scraping
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
crwlrsoft/crawler
Library for Rapid (Web) Crawler and Scraper Development
Go-phie/gophie
An Aggregator Engine for searching and downloading movies free - NO ADs!
kennethreitz/requests-html
Pythonic HTML Parsing for Humans™
driscoll42/ebayMarketAnalyzer
Scrape all eBay sold listings to determine average/median pricing, plot listings over time with trend lines, and extract to excel
DiegoCaraballo/Email-extractor
The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
RyuzakiH/CloudflareSolverRe
Cloudflare Javascript & reCaptcha challenge (I'm Under Attack Mode or IUAM) solving / bypass .NET Standard library.
RuthGnz/SpyScrap
CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.
hridaydutta123/the-youtube-scraper
Download YouTube video description and video comments without using the YouTube API.
Bishalsarang/Leetcode-Questions-Scraper
Scrape Algorithm Questions from leetcode and generate html and epub file
johnbumgarner/newspaper3_usage_overview
This repository provides usage examples for the Python module Newspaper3k.
fedecalendino/nintendeals
Library with a set of tools for scraping information about Nintendo games and its prices across all regions (NA, EU and JP).
fernandod1/Instagram-to-discord
Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!
voliveirajr/seleniumcrawler
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
alash3al/scraply
Scraply a simple dom scraper to fetch information from any html based website
JusticeRage/ApkTrack
ApkTrack is an Android app which checks if updates for installed APKs are available.
yousefkotp/Movies-and-Series-Scraper
A console application to scrape a valid watching links for any movie or series with exact season and episode number, you can also download a whole season with one click.
html2rss/html2rss
📰 Build RSS 2.0 feeds from websites (and JSON APIs) automatically or with a few CSS selectors.
autogram-is/spidergram
Structural analysis tools for complex web sites
pavlovtech/WebReaper
Web scraper, crawler and parser in C#. Designed as simple, declarative and scalable web scraping solution.
maxmindlin/scout-lang
A web crawling programming language
Cartmanishere/zippyshare-scraper
A module to get direct downloadable links from zippyshare download page.