Split into packages and use nx

[JoosepAlviste]

Improvements for working with nx and monorepos:

• Add generator for creating new packages
• Update TypeScript to v5 so that we can use the "bundler" module resolution configuration.
• Set up and double check nx job caching for each script:
  • lint
  • build
  • tsc
  • test
  • test:coverage
  • test:watch
  • test:e2e
  • migrate and migrate:test, input is probably the migrations?
  • Codegen in FE depends on BE GraphQL schema
• Since we're bundling the API files when building, then we need to fix the various prod scripts: migrate:prod, start:prod, start:jobs:prod
• Look into distributed caching
  • Could do Nx cloud, we're probably in the free tier
  • Or to an AWS S3 bucket: https://github.com/bojanbass/nx-aws
• Have a base Docker image with the build dependencies that we can just extend
• Package.json linting with syncpack, we already have it installed but it probably doesn't make sense to run it in CI
• Try to use nx environment variable handling instead of our custom approach. This means that we'll have to run everything from the root of the project, through nx, though. Not sure how I feel about that...
  • https://nx.dev/recipes/tips-n-tricks/define-environment-variables#define-environment-variables
  • We'd need to remove .env.example and use a .env.local which wouldn't be tracked in Git.
  • Each script would define its configuration in nx.json (or project.json?), which would automatically include the .env file.
  • It kinda makes sense as the code itself shouldn't really be reading .env files, but using the environment variables that are given to it.
• Use a custom nx executor for deployment so that a package.json and a pnpm-lock.yaml file are created in the dist folder, including only the dependencies required for the deployed application (api or webapp). We're currently using pnpm deploy, but that's not taking full advantage of nx as we need to copy ALL files in the Dockerfile, not just the ones that are relevant to us. Maybe the nx way of deploying could improve this?
  • Not sure if this is worth changing as the current approach seems to work. But it's good to try these things out I suppose.
  • https://nx.dev/ci/recipes/other/ci-deployment#prepare-applications-for-deployment-via-ci
• Try building each package rather than exporting source files. We probably need declarationMap so that go to definition goes to the source, not the type definition in dist. This means that we could probably get rid of esbuild. We could also use absolute imports in packages.
  • This means that during dev mode we need to run watch on all the packages

Extract packages and apps:

• Prettier config
• ESLint config
• Type utils
• TypeScript config
• Logger, Pino configuration. Also start using it in the FE Fastify server.
• Split e2e tests to a separate project
• Split mock TMDB server to separate project
• UI components
  • We should have 1 theme in the UI package with the colors and basic values, and then 1 theme in the app, including the more specific variables
  • Maybe also 1 "base" theme?
• Factories
• Split background job to separate project, probably requires extracting some business logic to a package
• Formatting things
• Maybe some business logic

Current problems with the new set up:

• Duplicated small utility functions, gotta figure out a package for those (or multiple packages)
• Duplicated configuration files. All packages and projects need a .eslintrc.json, .prettierrc.json, tsconfig.json and so on
• Duplicated set up for tools like the logger, db connection, maybe some others?
• Duplicated Vitest config

Resources:

• Monorepo Tools
• [AskJS] Is there a silver bullet for consuming Typescript libraries in a Monorepo?
• TypeScript project references
• How to structure an Nx Workspace ?
• How to structure your Angular apps like a Googler

[JoosepAlviste]

I think that we can consider this done for now and create tasks out of the un-checked items if we want to implement any of them.