This is the source code for https://jsr.io, the new JavaScript registry.
Important
The rest of this README is only relevant to those interested in contributing to the jsr.io registry. If you are looking for information on how to use the registry, please see https://jsr.io/docs.
Goals
- Robust
- Low maintenance
- Cheap
- Open source
Implementation details
- Modules and package metadata are stored on Google Cloud Storage (GCS)
- npm compatibility tarballs are stored on Google Cloud Storage (GCS)
- Management API is implemented in Rust and runs on Google Cloud Run
- Frontend uses Fresh and is running on Google Cloud Run in 6 regions
- https://jsr.io, https://api.jsr.io, and https://npm.jsr.io are served by a
Google Cloud Load Balancer
- Google Cloud CDN is used for caching
- Module, package metadata, and npm tarballs is served directly from GCS
- /api requests are proxied to the management API
- All other requests are proxied to the frontend
- Data is stored in PostgreSQL (using Google Cloud SQL)
- The database is highly available
- Not used for serving registry requests
- Distributed tracing using Google Cloud Trace (and Jaeger in development)
If you are just interested in making changes to the frontend, you can run the frontend in a development mode that connects to the production API.
- Clone this repo
- Install Deno (https://deno.land/#installation)
- Add the following to your
/etc/hosts
127.0.0.1 jsr.test 127.0.0.1 api.jsr.test 127.0.0.1 npm.jsr.test
deno task prod:frontend
You can view the registry at http://jsr.test
. This frontend is connected to
the production API - use it with the same care that you would use the live
registry.
In this mode, you will run the frontend and the API locally. This is useful for making changes to the API.
-
Clone this repo
-
Install Deno (https://deno.land/#installation)
-
Install Rust (https://rustup.rs/)
-
Add the following to your
/etc/hosts
127.0.0.1 jsr.test 127.0.0.1 api.jsr.test 127.0.0.1 npm.jsr.test
-
Set up
api/.env
file:- For @denoland employees: Download the
.env
file from 1Password (it's namedjsr local .env
), and set upDATABASE_URL
to point to your local Postgres database. - For everyone else:
- Create a GitHub App (https://github.com/settings/apps/new)
- Callback URL: "http://jsr.test/login/callback"
- Check "Request user authorization (OAuth) during installation"
- Disable "Webhook"
- Set "Account permissions" > "Email addresses" to "Read-only"
- Copy
api/.env.example
toapi/.env
- Set
GITHUB_CLIENT_ID
andGITHUB_CLIENT_SECRET
to the values from the GitHub App you created in step 1. - Set
DATABASE_URL
to point to your local Postgres database.
- Create a GitHub App (https://github.com/settings/apps/new)
- For @denoland employees: Download the
-
Install
sqlx
by runningcargo install sqlx-cli
macOS
- Postgres installed and running:
brew install postgresql
- Postgres database created with
createdb registry
- Postgres user created and granted access to the database
- Run
cd api
- Run
cargo sqlx migrate run
- If you get the error
role "postgres" does not exist
, runcreateuser -s postgres
.
- If you get the error
Linux
docker
&docker-compose
installed and running- Run
cd api
- Run
cargo sqlx migrate run
- If you get the error
role "postgres" does not exist
, runcreateuser -s postgres
.
- If you get the error
deno task services:macos
ordeno task services:linux
in one terminaldeno task dev:api
in another terminaldeno task dev:frontend
in another terminal
You can view the registry at http://jsr.test
. The API can be found at
http://api.jsr.test
.
- Create a new directory with a
deno.json
cd
into that directory- Run
JSR_URL=http://jsr.test deno publish
It may be helpful to have a large variety of packages published to your local dev environment to simulate a live environment. The quickest way to fill the registry with data is to publish deno_std to the registry. This can be done via the following steps:
- Clone https://github.com/denoland/deno_std in the same parent folder as the
jsr
project - In the
deno_std
folder, rundeno run -A _tools/convert_to_workspace.ts
. - Run
JSR_URL=http://jsr.test deno publish
to publish all of the @std packages to your local dev environment.
- Run
psql registry
- Run
SELECT name,github_id from users;
- You should see a table with your name and GitHub ID. Copy your GitHub ID.
- Run
UPDATE users SET is_staff = true WHERE github_id = xxxxxxx;
, replacingxxxxxxx
with your copied GitHub ID from the previous step. - You should see a success message confirming one row has been updated.
When the database schema has been changed, you can migrate the local database by running this command:
cd api; sqlx migrate run
To load bad words into the database:
- Download https://cloud.google.com/sql/docs/postgres/sql-proxy
- Run in a terminal
cloud-sql-proxy -g [database connection string] -p 5433
- Create a
bad_words.sql
file, with the contents as:
INSERT INTO bad_words (word) VALUES
('word_1'),
-- more words
('word_2');
- In a separate terminal window run
psql postgres://127.0.0.1:5433/registry --user [your username] -f bad_words.sql
, and provide the password for the provided username.
During local dev, traces are sent to Jaeger. You can view them at
http://localhost:16686. You can find traces in API HTTP requests by inspecting
the x-deno-ray
header.