/npm-packages-deps-retrieval

Primary LanguageJavaScriptApache License 2.0Apache-2.0

npm-packages-deps-retrieval

Running

The requirements are: docker, docker-compose, make. To run the application, you need to clone the repository and run the following commands:

$ make docker-build
$ make start

Go to http://localhost:8000.

TODOs

  • Add diagram of the architecture
  • Add a way to observe components stats
  • Add tests
  • Add error message in web page
  • Dockerize application

Ideas

  • Use npm registry webhook to point to the dispatcher
  • Use a graph database that should be much more suitable for this data
  • Add an in-memory LRU cache to both workers and APIs
  • Add a distributed in-memory LRU cache (redis)
  • Before hitting the DB it may worth adding a bloom filter on top
  • For each package with unpinned dependencies, consider each compatible ones

Limitations

  • Dependencies prefixed with ~ ^ are threated as pinned
  • Dependencies with different qualifiers (>=, <, etc.) are ignored