web-scraping-python
There are 201 repositories under web-scraping-python topic.
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
seleniumbase/SeleniumBase
Python APIs for web automation, testing, and bypassing bot-detection.
D4Vinci/Scrapling
Undetectable, Lightning-Fast, and Adaptive Web Scraping for Python
omkarcloud/botasaurus
The All in One Framework to build Awesome Scrapers.
tinyfish-io/agentql
AgentQL is an AI-powered query language for web scraping and automation. It uses natural language selectors to find data on any page, including authenticated content. AgentQL queries are self-healing as UI changes and work across similar sites. Users can define structured data output, making AgentQL versatile for developers and data scientists.
scrapfly/scrapfly-scrapers
Scalable Python web scraping scripts for +40 popular domains
davidteather/everything-web-scraping
Learn everything web scraping with David Teather Codes on YouTube
oxylabs/Python-Web-Scraping-Tutorial
In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex.
drshahizan/python-web
This topic explains how to implement web scraping and python web development. Web scraping topics such as scrapy, beautiful soup, and others will be covered. A case study based on a Malaysian website.
DataCrawl-AI/datacrawl
A simple and easy to use web crawler for Python
mike-gee/webtranspose
Web scraping API for building AI applications.
GoncaloMark/CobWeb-lnx
CobWeb is a Python library for web scraping. The library consists of two classes: Spider and Scraper.
vishwajeetdabholkar/eGet-Crawler-for-ai
Web scraping framework built for AI applications. Extract clean, structured content from any website with dynamic content handling, markdown conversion, and intelligent crawling capabilities. Perfect for RAG applications and AI training data pipelines. Features async processing, browser management, and Prometheus monitoring.
Smartproxy/Python-scraper-tutorial
A short introduction to scraping with Python with given steps and an example scraper script.
mhmdkardosha/CAT-Reloaded-2025-Data-Science-Roadmap
Roadmap for Data Science circle associated with CAT Reloaded.
Narenpradhan/WatchTower
WatchTower - A platform to save your valuable time while staying updated in the Cyber realm.
PB2204/Covid-19
This Is A Web Scraping Projects With Covid-19 Data From 2 Very Popular & Authentic Websites
Elmehdi9/web-scraping-projects
This repository provides various web scraping projects in Jupyter notebooks for both learning and data-related workshopes
FirasKahlaoui/news-headlines-tracker
The News Headlines Tracker application collects the latest news headlines from major news sources such as CNN, BBC, and The New York Times.
shawnCaza/compodio
Putting the podcast in community radio
boo283/Facebook_comment_crawler
The Facebook Comments Crawler is an unofficial tool for extracting comments from Facebook posts using Selenium in Python. It's designed to aid in academic and personal research. #Facebook comments scaper #Facebook comments crawler
gayanukabulegoda/Web-Scraping-Starter-Kit
Repository designed to help freshers easily grasp the basics of web scripting, offering simple guides and examples to build a strong foundation.
odevjorge/instagram-post-fetcher
"instagram-post-fetcher" is a Python module leveraging Selenium to extract Instagram post details, including account username, descriptions, media URLs, and post timestamps. Simplifying access to Instagram data for analytics and research.
oxylabs/asynchronous-web-scraping-python
A comparison of asynchronous and synchronous web scraping methods with practical examples.
saksham-joshi/web_scrap_and_analysis
Extracted text from blogs by "insights.blackcoffer.com" using BeautifulSoup and sentiment is analyzed using pandas module.
TerranKartikTellus/Web-Scraping
All about scraping domains from the 'World Wide Web'
lombardo-luca/LePrAn
Letterboxd Profile Analyzer (LePrAn) is a simple tool to see statistics about your letterboxd.com profile.
omkarcloud/gitpod-selenium
Run Python Selenium in GitPod
Abhayparashar31/Web-Scraping-Projects
Drinking coffee is my second favorite thing to do, web scraping will always be first.
Morgscode/desktop-webpage-text-crawler
This desktop GUI will index, format and create .txt files from the text content from webpages you request, so long as HTML or JSON is sent as a response. You can crawl sites as single pages, crawl all internal links on a page, or crawl all links within the page's <nav> tag(s). You can also decide to extract only page titles, the main text content, or all text content from the page. The crawler has some built-in basic error logging.
Nayemjaman/mobilehouse
Scraped news data using Scrapy and make a pipeline to push data in PostgreSQL database.
oxylabs/web-scraping-google-sheets
Guide to Using Google Sheets for Basic Web Scraping
irfanalidv/trustpilot_scraper
A Python library for scraping Trustpilot reviews.
Mindful-AI-Assistants/SP2024-Election-Analysis
📊 An analysis of voting patterns in São Paulo's 2024 elections, focusing on voter behavior, absenteeism, and geographic trends.