benqian's Stars
THUDM/ChatGLM-6B
ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型
rasbt/LLMs-from-scratch
Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step
ultrafunkamsterdam/undetected-chromedriver
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
rmax/scrapy-redis
Redis-based components for Scrapy.
VeNoMouS/cloudscraper
A Python module to bypass Cloudflare's anti-bot page.
Anorov/cloudflare-scrape
A Python module to bypass Cloudflare's anti-bot page.
my8100/scrapydweb
Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:
DormyMo/SpiderKeeper
admin ui for scrapy/open source scrapinghub
fabienvauchelles/scrapoxy
Scrapoxy is a super proxy aggregator, allowing you to manage all proxies in one place 🎯, rather than spreading it across multiple scrapers 🕸️. It also smartly handles traffic routing 🔀 to minimize bans and increase success rates 🚀.
yifeikong/curl_cffi
Python binding for curl-impersonate via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
TheWebScrapingClub/webscraping-from-0-to-hero
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python
andrewnguonly/Lumos
A RAG LLM co-pilot for browsing the web, powered by local LLMs
nitefood/asn
ASN / RPKI validity / BGP stats / IPv4v6 / Prefix / URL / ASPath / Organization / IP reputation / IP geolocation / IP fingerprinting / Network recon / lookup API server / Web traceroute server
istresearch/scrapy-cluster
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
omkarcloud/botasaurus
The All in One Framework to build Awesome Scrapers.
alecxe/scrapy-fake-useragent
Random User-Agent middleware based on fake-useragent
selfteaching/How-To-Ask-Questions-The-Smart-Way
AccordBox/awesome-scrapy
A curated list of awesome packages, articles, and other cool resources from the Scrapy community.
mouday/spider-admin-pro
spider-admin-pro 一个集爬虫Scrapy+Scrapyd爬虫项目查看 和 爬虫任务定时调度的可视化管理工具,SpiderAdmin的升级版
AtuboDad/playwright_stealth
playwright stealth
ZNLP/BigTranslate
BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages
daijro/browserforge
🎭 Intelligent browser header & fingerprint generator
bennyschmidt/next-token-prediction
Next-token prediction in JavaScript — build fast language and diffusion models.
TheWebScrapingClub/TheScrapingClubFree
The Web Scraping Club Free Repository
roycehaynes/scrapy-rabbitmq
A RabbitMQ Scheduler for Scrapy
shengchenyang/AyugeSpiderTools
使 scrapy 开发不用在意 item,pipeline,middleware 等通用场景下模块的编写,解放开发者的双手。
ispras/scrapy-puppeteer
Library that helps use puppeteer in scrapy.
owen9825/captcha-middleware
A middleware layer for Scrapy that detects CAPTCHA tests and solves them
cuicaihao/Annotated-Transformer-English-to-Chinese-Translator
An "annotated" version of the Transformer Paper in the form of a line-by-line implementation to build an English-to-Chinese translator.
dsdanielpark/hf-transllm
LLMtranslator translates and generates text in multiple languages.