omarsagoo/makescraper_v1

🕷 makescraper

📚 Table of Contents

1.Description 2. Project Structure 3. Getting Started

Description

This web scraper grabs all of the articles and related articles off of the google news web page, stores it into JSON and adds the articles (without duplicates) to a database.

Project Structure

📂 makescraper
├── README.md
└── scrape.go

Getting Started

1. Create an empty repository to store this file.

2. Run each command line-by-line in your terminal to set up the project:

   $ git clone git@github.com:omarsagoo/makescraper_v1.git
   $ cd makescraper_v1
   $ git remote rm origin
   $ git remote add origin git@github.com:YOUR_GITHUB_USERNAME/YOUR_REPO_NAME.git
   $ go mod download

3. Run this command to start scraping:

   $ go build && ./makescraper_v1