data-scraping

There are 625 repositories under data-scraping topic.

  • web-scraping

    je-suis-tm/web-scraping

    Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

    Language:Python8332711185

  • ScriptSmith/reaper

    Social media scraping / data collection tool for the Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs

    Language:Python386261368

  • Voldrix/onlyfans-dl-2

    OnlyFans content downloader v2

    Language:Python2171311233

  • toki-plus/video-mover

    全自动短视频搬运工具，支持自动下载、去重、AI生成标题+标签、上传，可二开扩展至多平台，例如：TikTok->视频号/抖音/小红书、抖音->TikTok/视频号/小红书......video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.

    Language:Python21341

  • drshahizan/special-topic-data-engineering

    This course presents to the students recent research and industrial issues pertaining to data engineering, database systems and technologies. Various topics of interests that are directly or indirectly affecting or are being influenced by data engineering, database systems and technologies are explored and discussed.

    Language:Python12741380

  • zohaibbashir/Google-Maps-Scrapper

    This code is used to perform web scraping and data extraction from Google Maps. It is particularly designed for obtaining information about businesses, including their name, address, website, phone number, reviews, and more.

    Language:Python1065239

  • ohsusannamarie/Instant-Data-Scraper-Chrome-Extension-v0.1.7

    Instant Data Scraper packed Chrome extension v0.1.7 (WITH LinkedIn scraping functionality)

    Language:JavaScript733256

  • dddat1017/Scraping-Youtube-Comments

    Scrape comments from any Youtube video

    Language:Python711222

  • sushil-rgb/AmazonMe

    Introducing AmazonMe, a Python-based web scraper designed to extract data from amazon.com using the requests and beautifulSoup libraries. It simplifies navigation and makes it easy to gather information from Amazon’s website efficiently.

    Language:Python6621322

  • SwatiModi/e-commerce-web-scraper

    Scraping details of products from ecommerce websites using python

    Language:Python602113

  • MuhammadAmir5670/psx-data-reader

    A scraper for downloading Pakistan stock exchange's data into Python Pandas DataFrame.

    Language:Python574421

  • toki-plus/AB-Video-Deduplicator

    一款强大的Python视频去重GUI工具，采用高帧率抽帧混合算法，以规避短视频平台查重。支持GPU加速。video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.

    Language:Python557

  • deedy/cbse_schools_data

    Cleaned, scraped data of all 20,367 CBSE schools, primarily in India, in 2018. Data scraped from: cbseaff.nic.in/cbse_aff/schdir_Report/userview.aspx

    Language:Python503022

  • kb22/GitHub-User-Insights-using-API

    The project involves using the GitHub API using user authentication to fetch information such as commits and repositories for that specific user and store them as CSV files for data collection and analysis.

    Language:Jupyter Notebook453131

  • LakshyaKhatri/Bookshelf-Reader-API

    A browsable REST API for recognizing book spines in an image.

    Language:Python453718

  • lspahija/torchestrator

    Spin up Tor containers and then proxy HTTP requests via these Tor instances

    Language:Kotlin45408

  • serpapi/google-search-results-java

    Google Search Results JAVA API via SerpApi

    Language:Java459327

  • Atharva-Phatak/Analysing-Glassdoor-Jobs

    Data Analysis of Job Postings on Glassdoor.

    Language:Jupyter Notebook412111

  • simonw/disaster-data

    Data scraped by https://github.com/simonw/disaster-scrapers

    3710310

  • zero-to-mastery/Complete-Python-Developer-Manual

    Class notes for Andrei Neagoie's Complete Python Developer course through Zero to Mastery.

    Language:Jupyter Notebook378032

  • ccubc/ChargeUp

    optimizing locations of electric vehicle charging stations in the city of Toronto

    Language:Jupyter Notebook331111

  • furkankose/bilkent-scheduler

    Bilkent Scheduler is an open-source tool designed to assist students at Bilkent University in planning their course schedules.

    Language:JavaScript33223
  • Toronto-housing-price-prediction

    slavaspirin/Toronto-housing-price-prediction

    Building Toronto Housing dataset from scratch to predict real estate prices

    Language:Jupyter Notebook31229

  • sallamy2580/python-web-scrapping

    Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

    Language:Python30203

  • Decodo/eCommerce-Scraping-API

    eCommerce Scraping API code examples for Python, PHP and Node.js

    Language:PHP26004

  • Decodo/Web-Scraping-API

    Web Scraping API code examples for Python, PHP and Node.js

    Language:JavaScript25009

  • pinouche/investment_analysis

    DCA analysis on the s&p 500

    Language:Jupyter Notebook25207

  • rileynwong/spotify-analysis

    Data analysis on my monthly playlists

    Language:Jupyter Notebook25103

  • Vyary/exile-profit

    This script engages with APIs to collect data and conduct fundamental computations for identifying the optimal approach to generate profits within the game. It also comes with a preconfigured and continuously updating Google Sheet for seamless utilization.

    Language:Python25322

  • yuis-ice/jseval

    Evaluate JavaScript on a URL through headless Chrome browser.

    Language:JavaScript25201

  • chaitanyarahalkar/Financial-Info-Extractor

    Extract financial information in CSV format for companies compliant to the NSE

    Language:Python24128

  • lidimayra/raspagem-de-dados-fatec

    :notebook: Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí

    Language:Jupyter Notebook245722

  • somdeep/Statball

    Statball - Football soccer stats analyser from top 5 european leagues with data obtained by web scraping from Fbref and Statsbomb

    Language:C#23263

  • bilalahhmedd/ebay-products-scraper

    This is ebay products scrapper to scrap images and metadata details of products from ebay.com

    Language:Python22119

  • fdd4s/matterport-downloader

    Download Skybox panoramic photos of Matterport houses

    Language:PHP22412

  • yashrane/DiscordScraper

    Collects and Analyzes textual data scraped from a Discord chat server

    Language:Python22402