About

TL;DR ExpressJS, VueJS cookbook, with evergreen recipes and templates (CRUD, CI/CD, QA, Testing, Cloud container deployment, Web Components, ES Modules, etc.) to develop applications faster, while reducing the need for rewrite or refactoring due to changes in dependencies.

Latest Version 0.6.10 - Released 2022 January 20 1030 +8GMT

upgrade to react-router-dom 6, update list of useful VS Code plugins

Ask for help and recommend improvements here

Considerations for this project are similar to favv. The difference between them are:

this repo is more of a cookbook and recipes are constantly being improved and updated
favv backend is written in Python, they can be used to improve on each other

We do not use Typescript in most folders because TS people can understand JS easily, but JS-only people need to pickup TS and its ecosystem. However, we use Typescript for ReactJS and NestJS applications

Features

Folder

Description

Features

@es-labs/esm

resuable ES module codes

Available as npm package also

@es-labs/node

reusable common JS codes

Available as npm package also

docker-devenv

Docker containers
supporting local development

Mongodb, Mysql, Keycloak(SAML/OIDC, etc IDP), Kafka, Hashicorp Vault, Redis

docs

Documentation
To constantly improved/updated

- Main documentation
- Secrets documentation
- Deployment notes
- Kafka docs and code
- TCP Server docs and code

js-node/expressjs

Base ExpressJS application
(REST API, Websocket, etc)

- CORS, proxy middleware, helmet, error handling, logging, OpenAPI
- Objection ORM (removed), Knex, MongoDb, Relational DB data example, migration, seed, GraphQL, Redis
- Webpush & FCM push notification, Sendgrid email, Nexmo SMS, Telegram
- AgendaJS message queue
- Unit Test & Integration Test

js-node/expressjs/apps/app-template

Custom application (app-template)
built on Base ExpressJS application

- [config] app configs
- [controllers]
- [models]
- [openapi] OpenAPI yamls
- [routes] API routes (also websocket handlers)
- [services] services to startup/shutdown
- [tables] config tables for generic table crud (t4t)
- [tests] folder for tests
- [graphql-schema.js] application GraphQL codes

js-node/expressjs/public/demo-express

Frontend to test backend features

- GraphQL, File uploads, Signed URL file upload to GCP Storage, websockets, SSE, webworkers (frontend demo)
- JWT using RSA, JWT refresh token, token in HttpOnly cookies, GA OTP, role, Passport SAML, OIDC
- Github OAuth2 login (setup - https://www.sohamkamani.com/blog/javascript/2018-06-24-oauth-with-node-js)
- Fido & Webauthn

js-node/*

Other Backend applications

- TCP server (event/stream)
- Scalable Websocket with Redis Pubsub
- Serial server
- openapi-file-joiner combine OpenAPI files

js-node/wip

Work In Progress

- Kafka consumer/producer
- cron / long-running process example

js-node/nest-admin

https://www.udemy.com/course/react-nest-admin

example application (e-commerce) backend using NestJS, for use with js-web/react-admin

Nest Microservice

Reference

https://github.com/Denrox/nestjs-microservices-example
https://www.merixstudio.com/blog/microservice-nestjs/

js-node/nest-cqrs

TBD Implement CQRS

https://docs.nestjs.com/recipes/cqrs

js-web/solid

SolidJS SPA

WIP

js-web/vue-nobundler

Vue 3 SPA no bundler + Bulma

- signed uploads, recaptcha
- Web component table, form & CRUD backend (files to note)

js-node/expressjs/apps/app-template/tables/	sample custom app table configurations
js-node/expressjs/router/t4t.js	handle backend CRUD API
@es-labs/esm/t4t-fe.js	frontend operations to interact with t4t.js
@es-labs/esm/t4t-validate.js	validation used by both front and backend
@es-labs/esm/bwc-table	used to display table
@es-labs/esm/bwc-t4t-form.js	form generated from table configurations
js-web/vue-nobundler/views/ui1.js	autcomplete, combobox, file upload example
js-web/vue-nobundler/views/ui2.js	table example
js-web/vue-nobundler/views/ui3.js	form example (with connection to backend)
js-web/vue-nobundler/views/ui4.js	table and form example (with connection to backend)

js-web/vue-vite

Vue 3 SPA using vite + Ant Design

- Leaflet Map, AntV Charts, PWA, Websockets, rxJS
- JWT refresh token, 2FA GA OTP, OIDC, SAML, Github OAuth
- Web Components (Webcam, Signature)
- Cypress Testing

js-web/react-admin

https://www.udemy.com/course/react-nest-admin

example application (e-commerce) frontend using ReactJS, for use with js-node/nest-admin

example-webpack
(Deprecated & removed)
last updated version

Vue 2 SPA using webpack + Vuetify

- Graphql, REST, VueCrudX, i18n, rxJS

.github/workflows

Github Actions (CI/CD)

- Manually Trigger
- On-push Trigger (WIP)

AMP Website

removed

QUICK START - ON YOUR LOCAL MACHINE

Requirements

Node 16+ LTS
Npm 8+ (using workspaces)
For Windows, integrate bash shell to cmd shell (when installing git), or use git-bash
Docker

Optional VS Code Plugins

NOTE Useful plugins if using VS Code:

Essentials
- Docker
- Live Server
- REST Client
- SSH FS
- vscode-database-client
Recommended
- SonarLint (requires java)
- GitLens
JS Language Specific
- es6-string-html
- ESLint
- Prettier (disabled)
- Vetur -> Volar (for VueJS)
NOT NEEDED
- thunder client (REST API)
- mongodb
- sqlite

Download

# clone repo and install backend
git clone https://github.com/ais-one/cookbook.git
cd cookbook

Installing & Updating Dependencies

Install dependencies for all workspaces!

Note

when doing npm i, it will always install latest version matching your package
sometimes you need to rebuild, delete all node_modules folders and the package-lock.json file in the root

# https://github.com/npm/cli/issues/708
# https://github.com/npm/cli/issues/2032
npm i --legacy-peer-deps # use this for now as there can be peer dependencies issues

Update dependencies for all workspaces!

# step 1
npm outdated # use this to check for outdated dependencies

# step 2
# manually change the versions in the package.json files

# step 3
npm i --legacy-peer-deps

# use npm ls to check on actual versions installed
npm ls

Install for single workspace

npm i @vscode/sqlite3 --workspace=@es-labs/node

npm i lorem-ipsum --workspace=@es-labs/node

ExpressJS Backend Setup & Run - development environment

npm run app --workspace=js-node/expressjs -- development

NOTES

MongoDB examples needs MongoDB to work. To resolve, chose one of the methods to install MongoDB in docs/backend/mongodb/install.md
The js-node/expressjs/apps/app-template/config/secret/*.env,js files are not present. So there maybe some console log errors (but it is ok to ignore) and websockets will not work. Quick start is still usable. Use the README.md to fill up

Run migration & app

# create and seed relational db on SQLite, (delete the dev.sqlite file each time before you run this)
# command: npm run knex -- <development / uat / production> <custom app name> <seed / migrate>
npm run knex --workspace=js-node/expressjs -- development app-template migrate
npm run knex --workspace=js-node/expressjs -- development app-template seed

# create and seed MongoDB requires MongoDB - you can skip this but MongoDB examples will not work
# command: npm run mongo -- <development / uat / production> <custom app name> <seed / update>
npm run mongo --workspace=js-node/expressjs -- development app-template seed

# run the backend
# command: npm run app --workspace=js-node/expressjs -- <development / uat / production> <custom app name, default = app-template>

# app name implied (implied as app-template if not in env)
npm run app --workspace=js-node/expressjs -- development

# or app name specified
npm run app --workspace=js-node/expressjs -- development app-template

# to include eslint checks
npm run app:lint --workspace=js-node/expressjs -- development app-template

Visit the following URLs

http://127.0.0.1:3000/api/healthcheck - app is running normally
http://127.0.0.1:3000 - Website served by Express with functional samples and demos
http://127.0.0.1:3000/api-docs - OpenAPI documentation

Note: to generate api docs, visit js-node/openapi-file-joiner and follow readme file, also look at the config properties OPENAPI_PATH and OPENAPI_VALIDATOR in js-node/expressjs/apps/app-template/config/common.env.js.

No bundler frontend

See js-web/vue-nobundler. Served from http://127.0.0.1:3000/native/index.html

Testing

To run unit & integration test on /api/categories endpoint. E2E testing is Work In Progress
TO TEST EVERYTHING PLEASE change describe.only(...) to describe(...) in the test scripts in js-node/expressjs/apps/app-template/tests

# run in development only
npm run test --workspace=js-node/expressjs

Long Running Processes

For long running processes such as tcp server (event mode, streaming mode), serial server, kafka producer, consumer, cron-triggered process, etc.

See js-node/README.md

Vite SPA Setup & Run - development environment

See js-web/vue-vite/README.md. To setup the configuration files. End-to-end testing example using cypress is here also.
Run the following

npm run dev --workspace=js-web/vue-vite

Visit

http://127.0.0.1:8080/ to view application
http://127.0.0.1:8080/nested/index.html to view another page (vite serving multi page, each page can be an SPA)

See js-web/vue-vite/README.md for more information on the vue-vite package

Why No SSR or SSG

potential slow rendering by server app, added complexity in code, rehydration errors, added complexity in server
nuxt/nuxt#8102
prefer static sites and lazy loaded SPA for now

SAML, OIDC, OAuth

Refer to link below on how to try out...

Keycloak README.md
Refer also to the following files
- js-node/expressjs/router/saml.js
- js-node/expressjs/router/oidc.js
- js-node/expressjs/router/oauth.js requires setup of github account and config setup here
You can test out on the js-web/vue-vite Signin UI,
- Use the following username / password credentials
  - for simple Login...
    - just clock on Login button, credentials test / test is already prefilled
    - then click on OTP button, the OTP 111111 is already prefilled
  - for SAML and OIDC... test-user / password
    - redirect to a keycloak IDP
  - for OAuth (Github), you need to configure your github settings (not recommended for starting out)
    - redirect to github login
  - for Mock user login, just click on Mock button

Fido2

Refer to following files for SPA sample (uses fido2-lib in backend)

Configuration

The js-node/expressjs/apps/app-template/config folder contains the config information.

You can override the configurations using <NODE_ENV>.env.js files, e.g. development.env.js or uat.env.js in the folder

NestJS and ReactJS E-commerce Application Example

NestJS Example

npm run start:dev --workspace=js-node/nest-admin # create the SQlite database called db.sqlite (in actual should be created using migration), shutdown server once it is fully up and running
npm run seed --workspace=js-node/nest-admin # seed the data
npm run start:dev --workspace=js-node/nest-admin # start the server again

Navigate to http://127.0.0.1:3000
You can use test.http file in the nest-admin folder file with VS Code REST Client plugin to test enpoints

ReactJS Example

npm run start --workspace=js-web/react-admin

Navigate to http://127.0.0.1:3001
login details: admin@test.com / password

Project Strcuture

+- .github/ : github related CICD and automations
+- .husky : git hooks
+- @es-labs/
|  +- esm/ : [shared es modules]
|     +- package.json
|  +- node/ : [shared cjs modules]
|     +- auth/ : authentication
|     +- comms/ : messaging
|     +- services/ : db, mq, etc.
|     +- config.default.js: defaults
|     +- config.js: config loader
|     +- package.json
|     +- traps.js
+- docker-devenv/ : docker for development environment
+- docs/ : documentation
+- js-node/ : nodejs applications (kafka, cron triggered, long running)
|  +- expressjs/ : express backend - See [js-node/expressjs/README.md](js-node/expressjs/README.md) for project structure
|  +- nest-admin/
|  +- openapi-file-joiner/ : pre-process utility to combine openapi yaml files for use in openapi related packages
|  +- serialserver/
|  +- tcpserver/
|  +- wip/ : projects in progress
|  +- worker-threads/ : demo on use of worker threads
+- js-web
|  +- react-admin/
|  +- solid/
|  +- vue-nobundler/ : frontend (Vue3 no bundler) - See [js-web/vue-nobundler/README.md](js-web/vue-nobundler/README.md) for Project Structure
|  +- vue-vite/: frontend (Vue3 rollup) - See [js-web/vue-vite/README.md](js-web/vue-nobundler/README.md) for Project Structure
+- .editorconfig
+- .gitignore
+- BACKLOG.md
+- CHANGELOG.md
+- LICENCE
+- package.json
+- README.md

CI/CD & Cloud Deployment

Cloud Services

The following Google Cloud Platform (GCP) services are used

Container Registry
Cloud Run
Cloud Storage
Mongo Atlas (hosted on GCP)

Refer to doc/deployment/home.md for documentation on deployments

Deployment Using Github Actions

.github/workflows/manual-gcp-expressjs.yml (Manually deploy js-node/expressjs to GCP CloudRun)
- selectable inputs
  - environment (uat for now, development does not deploy anything)
  - service (default = app-template)
  - branch
.github/workflows/manual-gcp-vue-vite.yml (Manually deploy js-web/vue-vite to GCP Cloud Storage)
- selectable inputs
  - environment (uat for now, development does not deploy anything)
  - csutom_app (to be implemented)
  - branch
.github/workflows/manual-gh-pages.yml (Manually deploy js-web/vue-vite to Github Pages)
- selectable inputs
  - environment (uat for now, development does not deploy anything)
  - csutom_app (to be implemented)
  - branch

NOTE config/secret contents will not be in repo for CI/CD (so you can get errors), those should be put in VAULT

Current secrets

GCP_PROJECT_ID
GCP_SA_KEY
VAULT_uat, passed in as VAULT

# do not merge
VAULT="unused"

# connect to a hashicorp vault and obtain secrets to merge
VAULT={ url, token } # base64 encoded

 # pass in secrets, this way is insecure and not a good way to send in secrets
VAULT={ secrets: { ... all your secrets here } } # base64 encoded

VERSION CHANGE NOTES

v0.6+ Improve organization, graceful exit, logging, project rename, add more nodejs applications, repo name vue-crud-x changed to cookbook
v0.5+ Improve organization and authentication, add new features
v0.4+ Improve folders and structure organization, handle CI/CD better
v0.3+ Reorganize folders and structure, for ease of developing and maintaining multiple applications.
v0.2+ uses Vuetify 2. Due to breaking changes from Vuetify 1 to 2, CRUD component code was refactored to be more UI framework agnostic (reduce dependencies!), easier to use, improving code quality, documentation and supprting article - VueJS+ExpressJS CRUD & Cookbook
v0.1 and Vuetify 1 will be supported under the v1 branch. You can refer to the v1 Legacy Article (For Historical Reference)

luke0623/cookbook

About

Features

QUICK START - ON YOUR LOCAL MACHINE

Requirements

Optional VS Code Plugins

Download

Installing & Updating Dependencies

Install for single workspace

ExpressJS Backend Setup & Run - development environment

Run migration & app

No bundler frontend

Testing

Long Running Processes

Vite SPA Setup & Run - development environment

Why No SSR or SSG

SAML, OIDC, OAuth

Fido2

Configuration

NestJS and ReactJS E-commerce Application Example

NestJS Example

ReactJS Example

Project Strcuture

CI/CD & Cloud Deployment

Cloud Services

Deployment Using Github Actions

VERSION CHANGE NOTES