web-scraping-python

There are 315 repositories under web-scraping-python topic.

  • zameen-com-scrapper

    Language:Python4
  • oxylabs-ai-studio-js

    oxylabs-ai-studio-js

    Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM agents with fresh data. AI Studio JS SDK for intelligent web data gathering.

    Language:TypeScript6

  • Web_Scraping_Python

    This repository contains a Python program that scrapes product information (names, prices, ratings, etc.) from an e-commerce website and stores the data in a CSV file. A useful tool for data collection and analysis! 📊

    Language:Python6

  • Web-Scraping

    All about scraping domains from the 'World Wide Web'

    Language:Python6
  • compodio

    compodio

    Putting the podcast in community radio

    Language:Python6

  • Web-Scraping-Projects

    Drinking coffee is my second favorite thing to do, web scraping will always be first.

    Language:Python6

  • Crawl4ai-RAG-with-Local-LLM

    A tool for scraping web documentation using Crawl4AI, converting it to Markdown, and preparing it for integration with local LLMs (like Ollama) to enhance their knowledge for learning and "vibe coding" workflows.

    Language:Python5

  • gitpod-selenium

    Run Python Selenium in GitPod

    Language:Dockerfile5

  • python-website-change-detector

    This Python script, "Website Change Detector," is designed to monitor specific websites for content changes.

    Language:Python5

  • mobilehouse

    Scraped news data using Scrapy and make a pipeline to push data in PostgreSQL database.

    Language:Python5

  • HTML-Fetcher-Script

    A Python script that allows users to fetch and optionally save the HTML content from a specified URL using `requests` library.

    Language:Python4

  • curl-with-python

    Master cURL in Python by using the PycURL library. Learn to send GET and POST requests, custom HTTP headers, and how to fix common problems.

    Language:Python4

  • gitpod-botasaurus

    Run Botasaurus in GitPod

    Language:Dockerfile4

  • recover-youtube-playlists

    Recover your playlists removed by YouTube's community guidelines bots

    Language:Python4

  • Source-Code-Viewer

    Online Source Code Viewer (get HTML source code from URL)

    Language:Python4

  • WebScrapideo

    Text Summarizer, Flipkart Web Scraper and Online Video Downloader

    Language:JavaScript4

  • yelp-scraper-scrapy-python

    Yelp Restaurant data scraping using python, scrapy spider

    Language:Python4

  • pyball

    pyball - python library for obtaining baseball statistics

    Language:Python4

  • Daraz-WebScraper

    🔥 Daraz Scraper – Extract product data, prices, ratings & images from Daraz with Python & Playwright. Export to Excel or MongoDB effortlessly! 🚀

    Language:Python3

  • web-crawler-openalex-semantic-research-papers-public

    Full-stack FastAPI + React app to search, filter, and analyze papers from OpenAlex & Semantic Scholar. Features charts, bookmarks, CSV export, and advanced filters for streamlined academic research.

    3

  • Selenium-Mini-Project

    A mini project using Selenium to scrape product data from Croma and visualize insights with Power BI.

    Language:Jupyter Notebook3

  • dork-seeker

    Simple Automatizated Google dorker script written in python

    Language:Python3

  • twitter-trends-fetcher

    A flask application that scrapes the top 5 twitter trends using Selenium and Proxymesh

    Language:Python3
  • ScrapyPy

    ScrapyPy

    ScrapyPy is a free, open-source, and powerful web scraping tool that simplifies the web scraping process

    Language:Python3

  • spider-course

    《深入了解Python爬虫攻防》课程课件及相关代码:大部分爬虫教程都是教一些基础或者是直接找一些案例讲解，已经入门但未熟练的人难以找到适合的课程及练习网站；只教人爬不教原理，以至于部分人学完还是知其然不知其所以然，无法灵活应用；而且很多课程掺杂了大量Python基础语法等内容充集数、知识点不连贯或者避重就轻等。 本课程以横向教学为主，介绍爬虫实际工作中用到的技术、思路及工具，并且以边开发网页边爬取的方式逐步深入爬虫与反爬虫的攻防知识，知己知彼。

    Language:HTML3

  • Spam-Message-Checker

    Checks for spam Messages and Dangerous URLs in the Message using Machine Learning Algorithm

    Language:Python3

  • mercedes-benz-dealership-scraper

    An end-to-end Python project that scrapes car dealership data from Cars.com, conducts data analysis and visualization, and provides insights into Mercedes-Benz vehicles in the market.

    Language:Jupyter Notebook3

  • google-serp-scraper

    Google SERP scraper

    3

  • gli99

    Web scraper for gifcities.org

    Language:Python3
  • imdb_web-scraping

    imdb_web-scraping

    Scraping Multiples pages of IMDB at a time to fetch top 250 movies data, sorted by user rating

    Language:Jupyter Notebook3
  • parse-html-pyquery

    parse-html-pyquery

    Learn to parse HTML using PyQuery, a Python library for web scraping and manipulating HTML.

    Language:Python3

  • JapaneseDataHorder

    Data-scraper for various japanese learning tools(Takoboto, Tatoeba, Ichimoe and OJAD)

    Language:Python3

  • Py_Web_Scrape

    A handy Web Scraping tool using python

    Language:Python3

  • DevDiv

    A comprehensive platform that aggregates headlines from various sources and allows users to post their own news. Our site employs a powerful web scraping mechanism to extract and clean data from external links, storing it securely in a database. With the convenience of a Django command, performing scheduled scraping tasks via cron jobs.

    Language:CSS3

  • WebScrapper

    web scrapping with selenium using chrome driver

    Language:Python3

  • local-leads-finder

    Local Leads Finder helps you uncover nearby business prospects in minutes, enter a keyword and city, watch real-time progress, and download clean lead lists ready for outreach. Perfect for agencies, freelancers, and growth teams who need consistent, enriched local data without the heavy work.

    Language:Python2