caiomajdalani/libquality

LibQuality API

Venturus

LibQuality API

---

Dependencies

• NodeJS 12.16.0 LTS (https://nodejs.org/en/)
• MySQL 5.7.29 +

Start

• For start on a local environment, you must have MySQL running on port 3306:

$ npm install pm2 -g
$ npm install 
$ npm run start:local

• For start on Docker environment:

$ npm run up:dev

IMPORTANT

• Ensure that you create a ".env" file with your personal access token to "GITHUB_TOKEN" variable. The API uses it at "Authorization" headers.

{ 
  'Authorization': `token ${process.env.GITHUB_TOKEN}` 
}

(https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token)

Docs

• To see swagger docs:

  • http://localhost:3000/api-docs

Tests

• To test and see the coverage on CLI:

$ npm test

• If will want to see more details about the coverage, open the follow file in your Browser:

  • ./coverage/lcov-report/index.html

Frameworks/Libraries

• API
  • Joi - Http schema validations
  • Body-Parser - Body parsing middleware
  • Consign - Autoload scripts
  • Cors - A Connect/Express middleware to enable CORS.
  • Express - Web framework
  • Express-Swagger-Generator - A module to serve swagger files based on express-swaggerize-ui and Doctrine-File
  • Express-Winston - Winston middleware for Express
  • Helmet - Improves security of Express apps by setting various HTTP headers
  • Mathjs - A extensive math library for JavaScript and Node.js
  • Moment - A JavaScript date library for parsing, validating, manipulating, and formatting dates
  • Mysql2 - MySQL client for Node.js
  • Sequelize - Promise-based Node.js ORM for Postgres, MySQL, MariaDB, SQLite and Microsoft SQL Server
  • Traverson - A Hypermedia API/HATEOAS Client for Node.js and the Browser
  • Winston - A logger for just about everything
• DevOps
  • PM2 - Process Manager for Nodejs
  • Docker - Package and run application
• Unit Tests
  • Jest - Testing Framework
  • Sequelize-mock - A mocking interface designed for testing code that uses Sequelize.

Basic Project Structure

-----------------───────────────────── Root
├── docker/ ────────────────────────── Dockerfiles
├── ecosystem-config/ ──────────────── PM2 config files
├── src/ ───────────────────────────── API Source
│   ├── __tests__/ ─────────────────── Unit tests 
│   ├── controllers/ ───────────────── API Controllers
│   ├── database/ ──────────────────── MySQL cofig and other files
│   ├── docs/ ──────────────────────── Swagger docs
│   ├── middlewares/ ───────────────── Middlewares injections
│   ├── models/ ────────────────────── MySQL models
│   ├── routes/ ────────────────────── API Routes
│   ├── server/ ────────────────────── API Server
│   ├── services/ ──────────────────── External functions and helpers
│   ├── index.js ───────────────────── Main file
├── docker-compose.yml ─────────────── Docker Compose file
├── jest.config.js ─────────────────── Config file for Jest
├── jestSetup.js ───────────────────── Setup file for Jest
├── README.md                       

How it Works

1. The user send a GET request to "/search" with the "project" he wants to search and his "user" on query: (project only accepting "vue" or "react" for now)

$ curl --request GET '${HOST}:${PORT}/search?project=${project}&user=${user}'

2. The application will try to find data from this project on database.

• If there is no data from this project on database, the application will follow the step 3.
• If it finds anything, the application will now check if this data have more than 1 day
  • If the data have more than 1 day, the application will update this data to inactive on database, and follow the step 3
  • If the data have less than 1 day, the application will follow the step 6

3. The application will send a GET request for Github API with the following data:

$ curl --request GET 'https://api.github.com/search/issues?q=repo:${repository}+state:open&sort=created&order=desc&per_page=100'

• For this, the response from github is something like that:

  {
    "total_count": 527,
    "incomplete_results": false,
    "items": [ {...}, {...}, ... ]
  }

• The property "items" is a array of objects, with each one of them a issue. However, even with the "total_count" property, the github api only returns 100 (at max) items for each page. So, our application will identify the "follow" link no headers and continue to send requests to next pages until all are done to get all items.

4. After all reqs finish, the application will map all results and make the calculations needed for "avgAge" (Average Age in days) and "stdAge" (Standar Deviation in days) based on Today's Date.

• After that, the project data will be saved on "project" table on database.

- And a log of that search will be saved on "searchLog" table on MySQL

5. The application will return to user to user the informations requested on response body

  {
    "status": 200,
    "message": "OK",
    "result": {
        "repository": "vuejs/vue",
        "issues": 527,
        "avgAge": 547,
        "stdAge": 31
    }
  }

TODO

• Statistics route/controller
  • The day by day data is already on MySQL, we need to create a route to organize this data for user consumes.
• Integrated Unit Tests
  • Mock express, to test the start of application, routes and controllers
  • Mock Sequelize (MySQL), to test connections, repositories, models and queries
• Automated Tests
  • The main goal is to create automated tests with BDD, using libs like SuperTest, Mocha and Chai. This tests need to be user tests, and can entender on CI/CD to ensure to block deploy if some issue pass through unit tests during build.
• CI/CD (Jenkins, GitLab..)
• Redis
  • Used as a "cache" to remove the dependency from MySQL as fresh data. Also to ensure the application will not crash and can be continue to retrieve data for users if something happens with database.