This repository is the complete code for the tutorial on SosNoob.com : "How te build your own web scraper easily?" You need a web site data to build a database or an API with it ? This tuto is for you !
The program can be export as an ".exe" (so don't need to have any dependency installed) and generate your data into JSON files. You can also modify it to directly push data on your database / API. For this example I scrap Github's data, especially the result list of "Javascript" in Github's search area.
The program use pkg
dependency which allow you to export it as an executable for :
Windows
OSX
Linux
So you or your community don't need any dependency once it export (no node js).
Download : All versions are available here
This program was built to be very light full, it can run on almost every configurations.
In fact SosNoob scaper is a tool :
- 100% performance : Developed to be very light full
- Simple to use | Plug and use
An item result example
{
"name":"item name",
"url":"Item's link",
"labels":[ "label 1", "label 2", ["..."], "label n" ]
}
Sosnoob scraper is build with full javascript : NodeJs/Npm, javascript.
Don't hesitate to create an issue or ask a question on sosnoob.com comment thread.
Coming soon !