umairqadir97/news-aggregator-with-fastapi

News Aggregator boilerplate built for Reddit and NewsAPI, using (Python, FastApi, PyTest, MySQL)

News Aggregator with FastAPI

Boilerplate news aggregation application build for Reddit and NewsAPI. Main idea is to use lightweight development frameworks like FastAPI rather than heavy weight lifting with Django/ Flask. Further information can be found in Problem Statement document.


Request Demo · Report Bug · Request Feature

Table of Contents

• About the Project
  • Built With
• Getting Started
  • Prerequisites
  • Installation
• Roadmap
• Contributing
• License
• Contact

About The Project

The application is built in accordance with Problem Statement document. All of the features are implemented. All the contraints are implemented, along with project documentation and unit-tests.

What's Not Implemented

Current version of this project does not implement cache server or job queue to minimize performance dependence on external APIs.

Built With

• Python
• FastAPI

Getting Started

General Technique to Aggregate

1. Get list of registered APIs
2. Loop over APIs
3. Make JSON object of configurations for each API
4. Write JSON parsers for each API
5. Combine result in uniform format to display
6. Return results

Detailed Approach

Every Request to API should be async; put the request in job tracker, get results and push back responses

News Listing Endpoint:

api.route('/news')

def get_top_news():

1. Get all available APIs
2. Call each API for getting top 10 news in JSON; Listing Function for Each API
3. Aggregate news from all APIs, discard empty responses

News Search Endpoint

api.route('/news?query=bitcoin')

def get_search_results():

1. Get all available APIs
2. Call each API with search query to get top 10 results in JSON; Searching Function for Each API
3. Aggregate results from all APIs, discard empty responses

Other Helper Functions

1. News_API json parser for listing responses; should return only required fields; ["title", "link", "source"]
2. Reddit_API json parser for listing results

Steps to Add any New API

1. Make a new file for new-API in src/external_api/
2. Take src/external_api/sample_api.py as reference
3. Prepare API Mapping object
4. Write JSON parser for that API
5. Register your API in API_COLLECTION in src/api_helper.py

Prerequisites

To run this project, should install project dependencies:

1. Python3
2. pip
3. Intsall Python packages

Instructions to Run

1. Clone the repo

git clone https://github.com/umairqadir97/news-aggregator-with-fastapi.git

2. Open terminal in project folder

cd news-aggregator-with-fastapi

3. Install python packages

pip3 install -r requirements.txt

3. Create your configurations file src/config.py. Take src/sample_config.py as example

4. Run server

python3 api.py

You can run tests with:

pytest

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b amazing_features)
3. Commit your Changes (git commit -m 'Add some Amazing Features')
4. Push to the Branch (git push origin amazing_features)
5. Open a Pull Request

Contribution guidelines

1. Writing more unit tests
2. Code review
3. Feature Enhancement

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Muhammad Umair Qadir - Email

LinkedIn: LinkedIn