web-scraping-python
There are 315 repositories under web-scraping-python topic.
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
seleniumbase/SeleniumBase
Python APIs for web automation, testing, and bypassing bot-detection with ease.
D4Vinci/Scrapling
🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!
omkarcloud/botasaurus
The All in One Framework to Build Undefeatable Scrapers
oxylabs/how-to-scrape-amazon-product-data
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
oxylabs/oxylabs-ai-studio-py
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM agents with fresh data. AI Studio python SDK for intelligent web data gathering.
tinyfish-io/agentql
AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.
0x676e67/rnet
An ergonomic Python HTTP Client with TLS fingerprint
scrapfly/scrapfly-scrapers
Scalable Python web scraping scripts for +40 popular domains
davidteather/everything-web-scraping
Learn everything web scraping with David Teather Codes on YouTube
oxylabs/Python-Web-Scraping-Tutorial
In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex.
drshahizan/python-web
This topic explains how to implement web scraping and python web development. Web scraping topics such as scrapy, beautiful soup, and others will be covered. A case study based on a Malaysian website.
thewebscraping/tls-requests
TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.
oxylabs/web-scraping-google-sheets
Guide to Using Google Sheets for Basic Web Scraping
DataCrawl-AI/datacrawl
A simple and easy to use web crawler for Python
vishwajeetdabholkar/eGet-Crawler-for-ai
Web scraping framework built for AI applications. Extract clean, structured content from any website with dynamic content handling, markdown conversion, and intelligent crawling capabilities. Perfect for RAG applications and AI training data pipelines. Features async processing, browser management, and Prometheus monitoring.
mike-gee/webtranspose
Web scraping API for building AI applications.
GoncaloMark/CobWeb-lnx
CobWeb is a Python library for web scraping. The library consists of two classes: Spider and Scraper.
mhmdkardosha/CAT-Reloaded-2025-Data-Science-Roadmap
Roadmap for Data Science circle associated with CAT Reloaded.
Decodo/Python-scraper-tutorial
A short introduction to scraping with Python with given steps and an example scraper script.
raymelon/tagalog-dictionary-scraper
Builds a Tagalog dictionary by collecting Tagalog words from tagalog.pinoydictionary.com
kvcops/Deep-Research-using-Gemini-api
AI-powered deep research tool leveraging web scraping for cost-effective, comprehensive analysis. Open-source and API-cost free!
Narenpradhan/WatchTower
WatchTower - A platform to save your valuable time while staying updated in the Cyber realm.
PB2204/Covid-19
This Is A Web Scraping Projects With Covid-19 Data From 2 Very Popular & Authentic Websites
Elmehdi9/web-scraping-projects
This repository provides various web scraping projects in Jupyter notebooks for both learning and data-related workshopes
FirasKahlaoui/news-headlines-tracker
The News Headlines Tracker application collects the latest news headlines from major news sources such as CNN, BBC, and The New York Times.
irfanalidv/trustpilot_scraper
A Python library for scraping Trustpilot reviews.
sarperavci/kick-unofficial-api
🛡️ Unofficial Kick.com API wrapper with automatic bypass protection.
boo283/Facebook_comment_crawler
The Facebook Comments Crawler is an unofficial tool for extracting comments from Facebook posts using Selenium in Python. It's designed to aid in academic and personal research. #Facebook comments scaper #Facebook comments crawler
lombardo-luca/LePrAn
Letterboxd Profile Analyzer (LePrAn) is a simple tool to see statistics about your letterboxd.com profile.
gayanukabulegoda/Web-Scraping-Starter-Kit
Repository designed to help freshers easily grasp the basics of web scripting, offering simple guides and examples to build a strong foundation.
JaydeepAgravat/SmartCode
Scraping LeetCode data, analyzing for insights, crafting a user-friendly dashboard, and building a problem recommender for optimized problem-solving.
Mindful-AI-Assistants/SP2024-Election-Analysis
📊 An analysis of voting patterns in São Paulo's 2024 elections, focusing on voter behavior, absenteeism, and geographic trends.
odevjorge/instagram-post-fetcher
"instagram-post-fetcher" is a Python module leveraging Selenium to extract Instagram post details, including account username, descriptions, media URLs, and post timestamps. Simplifying access to Instagram data for analytics and research.
oxylabs/asynchronous-web-scraping-python
A comparison of asynchronous and synchronous web scraping methods with practical examples.