This was originally the EthereumJS VM repository. On Q1 2020 we brought some of its building blocks together to simplify development. Below you can find the packages included in this repository.
🚧 Please note that the master branch is updated on a daily basis, and to inspect code related to a specific package version, refer to the tags.
Detailed version can be seen on Codecov.io
To update the diagram above use this edit url and open a new PR with the changes. Be sure to update both the diagram and the edit url.
First, make sure you have the ethereum-tests git submodule, by running:
git submodule init
git submodule updateThis monorepo uses Lerna. It links the local packages together, making development a lot easier.
TLDR: Setup
npm install
npm run buildTLDR: To update dependencies and (re-)link packages
npm run bootstrap
npm run buildAbove is the quickest way to set you up. Going down the road, there are two sets of commands: project and package-specific commands. You can find them at ./package.json and ./packages/*/package.json, respectively. Here's a breakdown:
Adds dependencies listed in the root package. Also, it executes the bootstrap script described below, installing all sub-packages dependencies.
Installs dependencies for all sub-packages, and links them to create an integrated development environment.
Builds all monorepo packages by default. If a scope is provided, it will only build that particular package.
Scoped example, that will only build the VM package: npm run build -- --scope @ethereumjs/vm
Builds all local packages that the provided package depends on (e.g.: @ethereumjs/blockchain), and builds itself.
If no scope is provided, npm run build:tree, will build all sub-packages.
Removes root and packages node_modules directories, and other generated files, like coverage, dist and others. This is useful to run after changing branches, to have a clean slate to work with.
These scripts execute lint and lint:fix respectively, to all monorepo packages. Worth noting that there is a git hook in place that runs npm run lint for every git push. This check can be skipped using git push [command] --no-verify.
npm install from the package directory, it will remove all links to the local packages, pulling all dependencies from npm. Run npm install from the root only.
There's a set of rather standardized commands you will find in each package of this repository.
Uses TypeScript compiler to build source files. The resulting files can be found at packages/<name>/dist.
Runs whatever is on npm run test script, capturing testing coverage information. By the end, it displays a coverage table. Additional reports can be found at packages/<name>/coverage/.
Generates package documentation and saves them to ./packages/<name>/docs.
Checks code style according to the rules defined in ethereumjs-config.
Fixes code style according to the rules. Differently from npm run lint, this command actually writes to files.
Runs the package tests.
Note that the VM has several test scopes - refer to packages/vm/package.json for more info.
As this project is powered by Lerna, you can install it globally to enjoy lots more options. Refer to Lerna docs for additional commands.
npm install -g lernalerna runlerna exec
Hoisting is enabled so dependencies are moved to the root node_modules. lerna clean does not remove the root node_modules so for convenience you can use the project script npm run clean.
There are some ways you can link this repository packages to other projects before publishing. You can symlink dependencies with npm link <package>, or install packages from the filesystem using npm install <folder>. But they are subject to some externalities and most importantly with how your package manager handles the lifecycle of packages during installs.
Note: Git references do not work with monorepo setups out of the box due to the lack of directory traversal on the syntax. E.g.:
npm install git@github.com:ethereumjs/ethereumjs-monorepo.git
One way to fetch packages remotely from GitHub before publishing is using gitpkg.now.sh.
But there's a cleaner way to manage your dependencies using Verdaccio.
Verdaccio is an npm registry and proxy that can be of great help to test packages locally. Check out their Getting Started guide.
npm install
lerna exec "npm publish --registry http://localhost:4873 --ignore-scripts"
lerna exec "npm unpublish $LERNA_PACKAGE_NAME --registry http://localhost:4873 --force"
npm config set @ethereumjs:registry http://localhost:4873
npm config delete @ethereumjs:registry
Verdaccio is also set up in the e2e_tests CI workflow and provides a way to install @ethereumjs
packages at an arbitrary commit in an external real-world project and run their unit
tests with it. This testing strategy is borrowed from ethereum/solidity which checks latest Solidity
against OpenZeppelin and others to keep abreast of how local changes might affect critical projects
downstream from them.
Tests like this are:
- a pre-publication sanity check that discovers how @ethereumjs performs in the wild
- useful for catching problems which are difficult to anticipate
- exposed to failure for reasons outside of @ethereumjs's control, ex: when fixes here surface bugs in the target.
E2E tests are constructed by cloning a real world target and using npm or yarn to replace its existing @ethereumjs dependencies with the versions published to CI's ephemeral private npm registry.
In practice, complex projects might have several versions of @ethereumjs packages nested in
their dependency tree. It's important to coerce all of them to the virtually published versions
for the test to be valid. This can be done using Yarn's selective dependency resolutions feature.
The verdaccio publication step writes a json map of @ethereumjs package names and their
virtually published versions to resolutions.json in the root directory. This object can be
injected into the E2E target's package.json under the resolutions key and Yarn will install
new versions everywhere as expected.
See our organizational documentation for an introduction to EthereumJS as well as information on current standards and best practices.
If you want to join for work or do improvements on the libraries have a look at our contribution guidelines.