scraping-websites
There are 1722 repositories under scraping-websites topic.
MontFerret/ferret
Declarative web scraping
Anorov/cloudflare-scrape
A Python module to bypass Cloudflare's anti-bot page.
elixir-crawly/crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
AmmeySaini/Edu-Mail-Generator
Generate Free Edu Mail(s) within minutes
Python-World/Python_and_the_Web
Build Bots, Scrape a website or use an API to solve a problem.
csbun/thal
译文：Puppeteer 与 Chrome Headless —— 从入门到爬虫
slotix/dataflowkit
Extract structured data from web sites. Web sites scraping.
KTZgraph/sarenka
OSINT tool - gets data from services like shodan, censys etc. in one app
avidLearnerInProgress/python-automation-scripts
Simple yet powerful automation stuffs.
spekulatius/PHPScraper
A universal web-util for PHP.
josephlimtech/linkedin-profile-scraper-api
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON.
baptisteArno/tinking
🧶 Extract data from any website without code, just clicks.
unixfox/pupflare
A webpage proxy that request through Chromium (puppeteer) - can be used to bypass Cloudflare anti bot / anti ddos on any application (like curl)
lkuffo/web-scraping
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
crwlrsoft/crawler
Library for Rapid (Web) Crawler and Scraper Development
Go-phie/gophie
An Aggregator Engine for searching and downloading movies free - NO ADs!
kennethreitz/requests-html
Pythonic HTML Parsing for Humans™
driscoll42/ebayMarketAnalyzer
Scrape all eBay sold listings to determine average/median pricing, plot listings over time with trend lines, and extract to excel
DiegoCaraballo/Email-extractor
The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
RyuzakiH/CloudflareSolverRe
Cloudflare Javascript & reCaptcha challenge (I'm Under Attack Mode or IUAM) solving / bypass .NET Standard library.
RuthGnz/SpyScrap
CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.
Bishalsarang/Leetcode-Questions-Scraper
Scrape Algorithm Questions from leetcode and generate html and epub file
johnbumgarner/newspaper3_usage_overview
This repository provides usage examples for the Python module Newspaper3k.
fedecalendino/nintendeals
Library with a set of tools for scraping information about Nintendo games and its prices across all regions (NA, EU and JP).
hridaydutta123/the-youtube-scraper
Download YouTube video description and video comments without using the YouTube API.
voliveirajr/seleniumcrawler
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
alash3al/scraply
Scraply a simple dom scraper to fetch information from any html based website
fernandod1/Instagram-to-discord
Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!
JusticeRage/ApkTrack
ApkTrack is an Android app which checks if updates for installed APKs are available.
yousefkotp/Movies-and-Series-Scraper
A console application to scrape a valid watching links for any movie or series with exact season and episode number, you can also download a whole season with one click.
html2rss/html2rss
📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.
Cartmanishere/zippyshare-scraper
A module to get direct downloadable links from zippyshare download page.
autogram-is/spidergram
Structural analysis tools for complex web sites
pavlovtech/WebReaper
Web scraper, crawler and parser in C#. Designed as simple, declarative and scalable web scraping solution.
Zhihan1996/TradeTheEvent
Implementation of "Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading." In Findings of ACL2021
alpdias/instagram-bot
🤖 Python bot to view stories, like and comment on Instagram