AlwaysUpdate ~ Web Crawler and Scraper 📰

AlwaysUpdate is an e-NewsPaper from Argentina, Colombia, Venezuela and Mexico, that update its news every day.

Getting started 🚀

virtualenv venv --python=python.3.7
source venv/bin/activate

pip install -r requirements.txt

export API_URL="$DATASCIENCE_API_HOST/api/v1/"
export GOOGLE_APPLICATION_CREDENTIALS="credentials.json"

You can execute the crawler with a POST request, in that case you must start the uvicorn server:

cd news_crawler_scraper
uvicorn app.main:app --reload

If you don't want to work with the server you can use:

python go_spyder_$JOURNAL_NAME.py

Journals:

Pull requests are welcome!. And if you have an idea for a feature and dont have time to do this, feel free to open a issue!