/Scrappy

Scraping Framework in C#

Primary LanguageC#MIT LicenseMIT

Scrappy Logo Scrappy

If you want to automate tasks in the web, this is your go to library. Navigate through websites with ease, fill out forms comfortably from your couch, let Scrappy do all the dirty work and get you what you want.

Features:

  • Form-Handling
  • Link-Handling
  • Header-Mimicking
  • HTML-Parsing and CSS-Selectors using CsQuery

####Usage####

How to fetch your IP from whatsmyip.org:

var browser = new Browser();

var page = await browser.Open("http://www.whatsmyip.org/");

var myIp = page.Select("#ip").Text();

####Samples####

Samples are probably outdated as the web is quite volatile!

  • WolframCalculator: Commandline tool which sends an expression to wolframalpha.com and displays the plain-text result to the user.
  • Tests: Check out the tests for Scrappy to see what can be done with it.

####Planned Features####

  • Resource downloader and management
  • Reactive extensions
  • Javascript interpreter

####Continous Integration####

scrappy MyGet Build Status

Get the latest build from this NuGet feed: https://www.myget.org/F/scrappy/api/v2/