This repository handles the backend of the web platform Manguru (Github).
It provides a RESTful API using NodeJS and Express.
To build and run this app locally, you will need a few things:
/!\ Important /!\ - Be sure to read the Development rules part before doing any update on the project resources!
Note - To run the tests, please refer to the associated part.
Please ensure that Docker is running on your computer before doing anything.
- Clone the repository
git clone --depth=1 https://github.com/lheydel/manguru-back.git manguru-back
- Install dependencies
cd manguru-back
npm install
- Build and run the project
docker-compose up -d --build
- Stop the project
docker-compose down
When the app is running, you can try making requests to http://localhost:8080 with tools like Postman, or directly using the Frontend part of the app.
You can also access your local database via http://localhost:29018 with tools like Robo 3T for example.
Note - In dev mode, the app is refreshing itself each time a typescript source file is updated, so you don't need to go through those steps again and again.
This part is mainly informative and quickly describes the main tools and technologies used in this project.
As the goal of manguru-back is to provide a server-side stateless RESTful API, it uses NodeJS, with the Express framework to simplify the management of the coming HTTP requests.
Also, for a better code structure and development efficiency, Typescript is installed and configured for all the parts of the project.
This project connects to a document oriented database managed by MongoDB, known for its high flexibility and adaptability.
Also, to facilitate the interaction with this database, MikroORM is used as a middleware between NodeJS and MongoDB. It provides an API to interact with the data within the database based on the definition of the entities of the project.
Note - When the app is running on your machine, you can access the associated MongoDB instance on
http://localhost:27018.
The test runner used in this project is called Jest. It manages unit, integration and end-to-end tests and works well with asynchroneous operations.
/!\ Important - The tests use your local database at http://localhost:27017. Make sure to have a MongoDB instance running.
To improve and maintain the global quality of the source code, this project uses a static code analysis tool named Sonarqube, which scan the project and detect code smells and security issues. It also get the test coverage stats from Jest.
This project is configured to trigger a build on Travis CI each time a push is done on this Github repository.
Each Travis build runs the test suite with Jest and scan the project with Sonarqube.
Also, for some branches, it builds a Docker image and push it on a private Docker repository. This image will then be used as part of the deployment of the entire platform.
Here are the rules you must follow if you want to make any code change on this project. Any modification that does not respect those rules will be rejected.
In order to maintain a good overall quality, every update must be supported by relevant tests and have a good evaluation on Sonarqube.
Also, please respect the general structure of the project to avoid any confusion.
As said before, the link between NodeJS and MongoDB is done by the middleware called MikroORM. It provides an API to access the database. Both the API and the database are built and updated from the project entities declarations.
MikroORM make managing and accessing the data much easier, but has an important drawback which must be taken care of before starting a project. Indeed, whenever you want to change the structure of the database, you have to manage the migration of the data all by yourself, either by doing it manually, or by including a dedicated service in your code base. MikroORM does not provide any script management for this task (at least at the time this was written).
As it is annoying to do it manually, a solution has been setup in the project to make our life easier. This solution is based on the concept of Version Struct.
Note - To illustrate the concepts described below, you can check the migration of the entity
Userfrom VS 1 to VS 2. The migration in itself doesn't have much interest since it was used as an example during the reflection behind this solution. You can see the details of the changes at/src/user/user.migrations.ts > _fromVs1To2()
Give to Caesar what belongs to Caesar - I did not come up with this solution all by myself. It is inspired from a similar situation I encountered in an internship at Atheris Services in 2018.
A Version Struct (or VS) merely designate the version of the structure of an entity. Each and every entity stored in the database must have a property called vs, which indicate its version number.
To update and access the latest VS available for an entity, refer to the enum VersionStruct in the file /src/common/properties.ts.
Example - If you want to change the structure of the entity User for the first time, you will update the value of
VersionStruct.USERfrom1to2.
First things first, if you want to update an entity in the database, you will have to update the entity model.
There are roughly three case scenarios when updating an entity. You can either add, remove or edit one or more properties.
To create a new property, there is no special action to do beside adding it to the model with the annotation @Property().
Example:
@Property()
username: string;If the new property depends on other data of the entity, please refer to Data migrations.
Removing a property is a delicate operation, as it causes a loss of data, which can't be migrated afterwards to another location since they don't exist anymore. As such, removing a property is prohibited.
Instead, you can use the @deprecated annotation to signal the deprecation of a property. This annotation can be used in the JSdoc of the property in the entity model. Also, dont forget to make this property optional.
Note - When you deprecate an attribute, please indicate the date (with the format
YYYY/MM/DD) and VS of deprecation.
Example - Let's say you have an entity
Userand you want to remove itsnameproperty:/** @deprecated 2019/07/05 - vs2 */ name?: string;With this annotation, we can know that
namehas been deprecated since the VS 2, at the date 2019/07/05.
Editing a property can mean many things. You can rename it, split it in multiple other properties, assemble several properties into one, etc..
However, all thoses operations are just a succession of adding and removing properties. Thus, you can do it by following the two previous parts. Just remember to write the details of the updates in the comments of the @deprecated annotations and to migrate your data.
Example - Let's say you have an entity
Userand you want to rename itsnameproperty intousername.
Entity model:/** @deprecated 2019/07/05 - vs2 - use [username] property instead */ name?: string;
Changing the data structure obviously doesn't mean the data will adapt themselves by using some weird magic. We need some migrator services to upgrade them to the new strucure.
A migrator is a service dedicated to the migrations of a single entity. It implements the interface BaseMigrator which provides a set of standard functions.
For each new VS, a function _fromVsXToY() must be created. This function gives the actual instructions to migrate the data of an entity type from the VS X to the VS Y (with Y = X + 1). This function is then called in upgradeVersionStruct() which take an entity as parameter.
Note - Each migrator must have the mention
// tslint:disable: deprecation no-switch-case-fall-throughon top of the file to be able to use the deprecated properties and perform smooth migrations.
The migrators are then used by the MigrationManager defined in /src/migrations.ts, which is called at the start of the application.