Extension of boilerplate webscraper by Ania Kubow
My first time messing around with webscraping, but hopefully useful for people!
Currently I have spun up a website using Heroku. If this scales maybe we will use something else?
To provide a scraped collection of sites for activism, combined into one place
The aim is to be FUNCTIONAL! I'm not particularly concerned with a nice UI, however maybe I (or someone else) will add a frontend at some point, however I think currently its better to invest time into scraping more sites
You can run this project on your local machine. Just pull it down and do the following:
- Run the command above to install all the packages
- Now run the server:
npm run start-
Open http://localhost:8000/results with your browser to see the results of the scrape in your terminal.
-
Finally, open the index.html file in your browser to see the results showing up.
Feel free to submit a pull request! I am new to web-scraping, and not an expert in javascript, so feedback is always appreciated! :D
To add a site, scrape the event info from an organization into a JSON, then create a new page for it- have that page return the JSON of events.
Then, link that site on the main page!
Copyright (c) 2022 Ian Laffey
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*Translation: Ofcourse you can use this for you project! Just make sure to say where you got this from :)
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.