This is an open-source project showing gratitude and giving back to the community, this was created with the aim of learning and understanding best practices with small pieces of code and at the same time helping other people to learn collaboratively by not just talking but showing them how.
// Working on it
- Src
- config
- controllers
- models (All your )
- routes
- v1
- v2
- scripts
- services
- utils
- tests
- app.js
- server.js
- .env
- .env.example
- package.json
- package-lock.json
- nodemon.json
- readme.md
- .gitignore
- Make sure you have an account and key in
- copy .env.example to .env
- Replace/add the value/keys required
- Install Node v14.16.0 and restart the shell if current version is not Node v14
- Download the code to a directory
- cd to that directory
- Now setup the environment as mentioned above
- npm install
- npm run dev
- npm run test
- node --inspect src/server.js https://medium.com/@paul_irish/debugging-node-js-nightlies-with-chrome-devtools-7c4a1b95ae27#.pmqejrn8q
- npm run doc
- Express.js (Framework)
- Helmet (Security) (Take precuations via Header best practices)
- Cors (Security) (From where resources can be loaded)
- Jest (Testing framework) (Dev Dependency)
- PM2 (Process Manager) (running node smoothly in production)
- Nodemon () (watching changes in file for server restart)
- Eslint (Linting) (Help improves code quality)
- Winston (Logger) (for logging in the app)
- Dotenv (Security) (Helps with environment variable)
- Joi (validation module)
- mongoose (ODM) (makes database operations easier)
- argon2 (security/cryptography) (hashing password)
- celebrate (Middleware) (validation in routes)
- sentry (APM) (for application monitoring)
- JWT (for authentication)
- Supertest (for high level api testing)
- @babel/core & @babel/node (for babel helping node environment)
- @babel/preset-env (for targeting specific environments)
- cookie-parser as cookie helper
- aws-sdk for accessing the aws resources
- Multer for file uploads
- Multer-s3 for easy uploads to s3 with multer
- uuid for generating unique ids and related unique ids (v4 and v5)
- jsdoc for internal code documentation
- We generally use compression through Nginx which can improve overall performance of the system
- For Large Scale project, we can divide the project into component/module/features instead of technical role wise, which is easier to maintain for large team
- Body-parse depreciated in newer version
- We should save the logs in file (with some kind of rotation or upload on regular interval of time), or stream it to somewhere for production usage
- As we are using node v14, and configured package.json ("type": "module",) so we don't need to transpile the code anymore for ES6+ to ES5
- You could use the babel or webpack for custom module aliasing instead of using long relative path
https://javascript.plainenglish.io/nodejs-security-best-practices-542528f910 https://www.bacancytechnology.com/blog/24-best-practices-to-make-your-nodejs-application-secure
https://pm2.keymetrics.io/docs/usage/application-declaration/
https://dev.to/santypk4/bulletproof-node-js-project-architecture-4epf https://www.freecodecamp.org/news/how-to-write-a-production-ready-node-and-express-app-f214f0b17d8c/ https://github.com/goldbergyoni/nodebestpractices https://12factor.net/
- Using babel or webpack for module aliasing (https://medium.com/bootstart/why-you-should-use-babel-resolvers-210615fc41d)
- saving the logs to file
- Launch a Typescript version
- Better Custom error messages with Joi & Celeberate
- protection against csrf
- Making cookie securer and taking data from env/config
- Load different config values based on current node environment
- Making sure the application waits for mongodb connection before starting
- Configure Corsoption
- Snyk for vulnerability detection in packages
- Using socket.io for adding any one of the real time applications
- Ability to block user jwt with the help of redis
- Check header algorith for jwt security (https://dev.to/techschoolguru/why-paseto-is-better-than-jwt-for-token-based-authentication-1b0c)
- Add a nginx config file that shows (ssl, compression, static file output, and payload limitation)
- Adding CI/CD
- Using redis cache to make few of the important read faster (eg: Loading default jobs)
- Image manipulation before upload
- See if we can directly upload to s3 instead of going through nodejs
- Refactor the code structure for component base over tecnical role base
- Dockerization with CI/CD
- Profiling for performance test
- Add Paytm Payment Gateway Integration
- Push Notifications, SMS and Email Notifications as well
- Api documentation using swagger-ui-express and swagger-jsdoc
- Pagination has been optimized by using, estimatedDocumentCount for counting and then using promise all to run the queries parallely
- Application restarts when memory limit reaches a particular amount of memory mostly for concern related to unxpected memory leakes
- For Redis: https://console.upstash.com/
- For MongoDb: MongoDB Atlas
- For Node.js Server
- For APM: Sentry
- Polymorphic - (saving joins) (when records are almost similar for most of the part)
eg: Job post of different fields and different work type can have different assessment criteria or different fields
-
Attribute - (save index, join and records) (when the record have different but similar value ) (eg: release date of movies in different year, of physical spec of product)
-
Bucket - (saving index) (bucketing/grouping data and storing in a single record) (eg: IOT storing 1 hour data in each record as array instead of 1 min of data in each record)
-
Outlier - (overall optimisation) (just for few instances we do not modify the model)
eg: books purchase by customer storing in book collection with flag to fetch from books collection (not every book is a best seller)
- Computed - (Save CPU) (pre compute things for large records like ratings, top 10,)
eg: Company Rating
- Subset-pattern - (Save Ram) (store frequent used record in sub doc and rest in their own collection)
eg: Applications in job posts (15 applications in job post and the rest in job application collections)
- Extended - (save join queries) (if a small of part of other collection is frequent used but not frequently updated then duplicate it)
eg: Company name, logo in jobs collection
- Approximation - (saves cpu) (when slightly less precise data is okay, computes only after a certain number of inserts)
eg: population count
-
Tree
-
PreAllocation
-
Document Versioning
-
Schema Versioning adding schema_version in the schema
- https://www.mongodb.com/blog/post/building-with-patterns-the-subset-pattern
- https://www.mongodb.com/blog/post/building-with-patterns-a-summary
npm run doctor Using: snyk test
- The total no of ids that can be generated by UUID4 are (with 128 bit) 340,282,366,920,938,463,463,374,607,431,768,211,456 Three hundred forty undecillion, two hundred eighty two decillion, three hundred sixty six nonillion, nine hundred twenty octillion, nine hundred thirty eight septillion, four hundred sixty three sextillion, four hundred sixty three quintillion, three hundred seventy four quadrillion, six hundred seven trillion, four hundred thirty one billion, seven hundred sixty eight million, two hundred eleven thousand, four hundred fifty six
- Using MongoDb user id (object id) in jwt Reason: Adding Email are PII risk, and from MongoDb 3.4, only identifiable information from id is timestamp of document creation. Our application is also secure as the user can not access or modify data just based on the id