scrape
There are 543 repositories under scrape topic.
twintproject/twint
An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
alirezamika/autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Anorov/cloudflare-scrape
A Python module to bypass Cloudflare's anti-bot page.
microlinkhq/metascraper
Get unified metadata from websites using Open Graph, Microdata, RDFa, Twitter Cards, JSON-LD, HTML, and more.
trevorhobenshield/twitter-api-client
Implementation of X/Twitter v1, v2, and GraphQL APIs
d60/twikit
Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers, user info, images...
glebarez/cero
Scrape domain names from SSL certificates of arbitrary hosts
markowanga/stweet
Advanced python library to scrap Twitter (tweets, users) from unofficial API
austinoboyle/scrape-linkedin-selenium
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
ScriptSmith/instamancer
Scrape Instagram's API with Puppeteer
unixfox/pupflare
A webpage proxy that request through Chromium (puppeteer) - can be used to bypass Cloudflare anti bot / anti ddos on any application (like curl)
Anonyfox/elixir-scrape
Scrape any website, article or RSS/Atom Feed with ease!
ultralytics/google-images-download
Google/Bing Images Web Downloader
danieldotnl/ha-multiscrape
Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.
yaroslaff/nudecrawler
Crawl telegra.ph searching for nudes!
andrewstuart/goq
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library
JaredLGillespie/proxyscrape
Python library for retrieving free proxies (HTTP, HTTPS, SOCKS4, SOCKS5).
evyatarmeged/Humanoid
Node.js package to bypass CloudFlare's anti-bot JavaScript challenges
essamamdani/search-result-scraper-markdown
This project provides a powerful web scraping tool that fetches search results and converts them into Markdown format using FastAPI, SearXNG, and Browserless. It includes the capability to use proxies for web scraping and handles HTML content conversion to Markdown efficiently.
rocketlaunchr/google-search
scrape google search results
JMousqueton/ransomware.live
🏴☠️💰 Another Ransomware gang tracker
meetyan/raise
A simple (and unofficial) GitHub Trending client that lives in your menubar.
Jimut123/jimutmap
API to get enormous amount of high resolution satellite images from satellites.pro quickly through multi-threading! create map your own map dataset. Bringing data to Humans.
luengwaiban/instagram-python-scraper
A instagram scraper wrote in python. Similar to instagram-php-scraper.Usages are in example.py. Enjoy it!
tegridydev/auto-md
Convert Files / Folders / GitHub Repos Into AI / LLM-ready Files
html2rss/html2rss
📰 Build RSS 2.0 feeds from websites (and JSON APIs) automatically or with a few CSS selectors.
DrKain/scrape-youtube
A lightning fast package to scrape YouTube search results
badoux/goscraper
Golang pkg to quickly return a preview of a webpage (title/description/images)
fefit/visdom
A library use jQuery like API for html parsing & node selecting & node mutation, suitable for web scraping and html confusion.
serp-spider/core
:spider: The PHP SERP Spider - A search engine scraper
SilentDemonSD/FZBypassBot
A Elegant Fast Multi Threaded Bypass Bot for Bigger Deeds. Try Now !!
Pringleman83/SportsBook
A sports data scraping and analysis tool
warifp/Shopee-Scrape
Shopee Scrape is a tool that functions to collect data - the data needed, such as finding data from photos, prices, names, store locations and others.
ndgigliotti/shopify-spy
Extract structured data from Shopify websites.
drudge/n8n-nodes-puppeteer
n8n node for browser automation using Puppeteer