mbt

The most flexible build tool for monorepo

mbt is a built tool for monorepo.

Features

Differential Builds
Content Based Versioning
Build Dependency Management
Template Driven Deployments

Status

mbt is production ready. We try our best to maintain semver. Visit Github issues for support.

Install

curl -L -o /usr/local/bin/mbt [get the url for your target from the links below]
chmod +x /usr/local/bin/mbt

Releases

Stable

OS	Download
darwin x86_64
linux x86_64
windows

Dev Channel

OS	Download
darwin x86_64
linux x86_64
windows

Building Locally

You need cmake and pkg-config (latest of course is preferred)
Get the code go get github.com/mbtproject/mbt
Change to source directory cd $GOPATH/src/github.com/mbtproject/mbt if you haven't set $GOPATH, change it to ~/go which is the default place used by go get. See this for more information about $GOPATH
make build will build and run all unit tests
make install will install the binary in $GOPATH/bin Make sure $GOPATH/bin is in your path in order to execute the binary

About

In the context of mbt, term 'Module' is used to refer to a part of source tree that can be developed, built and released independently. Modules can be built with different programming languages and their native build tools.

For example, you could have Java modules built with Maven/Gradle, .NET modules built with MSBUILD and NodeJS modules built with npm scripts - all in one repository. The idea is, module developers should be able to use the build tools native to their stack.

Each module in a repository is stored in its own directory with a spec file called .mbt.yml. Presence of the spec file indicates mbt that the directory contains module and how to build it.

Spec File

.mbt.yml file must be in yaml and has the following structure.

name: module-a      # name of the module
build:              # list of build commands to execute in each platform
  darwin:
    cmd: npm        # build command
    args: [run, build]    # optional list of arguments to be passed when invoking the build command
  linux:
    cmd: npm
    args: [run, build]
  windows:
    cmd: npm
    args: [run, build]

In the preceding spec file, we are basically telling mbt that, module-a can be built by running npm run build on OSX, Linux and Windows. With this committed to the repository, we can start using mbt cli to build in several ways as shown below.

# Build the current master branch
mbt build branch

# Build the current branch
mbt build head

# Build specific commit
mbt build commit <commit sha>

# Build only the changes in your local workdir without commiting
mbt build local

# Build everything in the current workdir
mbt build local --all

# Build a specific branch
mbt build branch feature

# Build only the modules changed in a specific branch relative to another
mbt build pr --src feature --dst master

# Build only the modules changed between two commits
mbt build diff --from <commit sha> --to <commit sha>

Dependencies

Module Dependencies

One advantage of a single repo is that you can share code between multiple modules more effectively. Building this type of dependencies requires some thought. mbt provides an easy way to define dependencies between modules and automatically builds the impacted modules in topological order.

Dependencies are defined in .mbt.yml file under 'dependencies' property. It accepts an array of module names. For example, module-a could define a dependency on module-b as shown below, so that any time module-b is changed, build command for module-a is also executed.

name: module-a
dependencies: [module-b]

One example of where this could be useful is shared libraries. A shared library could be developed independently of its consumers. However, all consumers gets automatically built whenever the shared library is modified.

File Dependencies

File dependencies are useful in situations where a module should be built when a file(s) stored outside the module directory is modified. As an example, a build script that is used to build multiple modules could define a file dependency in order to trigger the build whenever there's a change in build.

File dependencies should specify the path of the file relative to the root of git repository.

name: module-a
fileDependencies:
  - scripts/build.sh

Build Environment

When mbt executes the build command specified for a module, it sets up several important attributes as environment variables. These variables can be used by the actual build tools in the process.

Variable Name	Description
MBT_MODULE_NAME	Name of the module
MBT_MODULE_VERSION	SHA1 hash calculated based on the content of the module directory and the content of any of its dependencies (recursively)
MBT_BUILD_COMMIT	Git commit SHA of the commit being built

In addition to the variables listed above, module properties (arbitrary list of key/value pairs described below) is also populated in the form of MBT_MODULE_PROPERTY_XXX where XXX denotes the key.

One useful scenario of these variables would be, a build command that produces a docker image. In that case, we could tag it with MBT_MODULE_VERSION so that the image produced as of a particular Git commit SHA can be identified accurately. (We will also discuss how this information can be used to automatically produce deployment artifacts later in this document)

Describing the Change

When working in a dense, modular codebase it is sometimes important to assess the impact of your changes to the overall system. Each mbt build command variation has a mbt describe counterpart to see what modules are going to to be built. For example, to list modules affected between two git branches, we could run:

mbt describe pr --src <source-branch-name> --dst <destination-branch-name>

Furthermore, you can specify --json flag to get the output of describe formatted in json so that it can be easily parsed and used by tools in the rest of your CD pipeline.

Going Beyond the Build

mbt produces a data structure based on the information presented in .mbt.yml files. That is what basically drives the build behind scenes. Since it contains information about all modules in a repository, it could also be useful for producing other assets such as deployment scripts. This can be achieved with mbt apply command. It gives us the ability to apply module information discovered by mbt to a go template stored within the repository.

As a simple but rather useless example of this can be illustrated with a template as shown below.

{{ $module := range .Modules }}
{{ $module.Name }}
{{ end }}

With this template committed into repo, we could run mbt apply in several useful ways to produce the list of module names in the repository.

# Apply the state of master branch
mbt apply branch --to <path to the template>

# Apply the state of another branch
mbt apply branch <branch-name> --to <path to the template>

# Apply the state as of a particular commit
mbt apply commit <git-commit-sha> --to <path to the template>

# Apply the state of local working directory (i.e. uncommitted work)
mbt apply local --to <path to the template>

Output of above commands written to stdout by default but can be directed to a file using --out argument.

It's hard to imagine useful template transformation scenarios with just the basic information about modules. To make it little bit more flexible, we add a couple user-defined properties the data structure used in template transformation. First one of them is called, .Env. This is a map of key/value pairs containing arbitrary environment variables provisioned for mbt command.

For example, running mbt command with an environment variable as shown below would make the key TARGET available with value PRODUCTION in .Env property.

TARGET=PRODUCTION mbt apply branch

Second property is available in each module and can be specified in .mbt.yml file.

name: module-a
properties:
  a: foo
  b: bar

module-a shown in above .mbt.yml snippet would make properties a and b available to templates via .Properties property as illustrated below.

{{ $module := range .Modules }}
{{ property $module "a" }} {{ property $module "b" }}
{{ end }}

More realistic example of this capability is demonstrated in this example repo. It generates docker images for two web applications hosted in nginx, pushes them to specified docker registry and generates a Kubernetes deployment manifest using mbt apply command.

Template Helpers

Following helper functions are available when writing templates.

Helper	Description
`module <name>`	Find the specified module from modules set discovered in the repo.
`property <module> <name>`	Find the specified property in the given module. Standard dot notation can be used to access nested properties (e.g. `a.b.c`).
`propertyOr <module> <name> <default>`	Find specified property in the given module or return the designated default value.
`contains <array> <item>`	Return true if the given item is present in the array.
`join <array> <format> <separator>`	Format each item in the array according the format specified and then join them with the specified separator.

These functions can be pipelined to simplify complex template expressions. Below is an example of emitting the word "foo" when module "app-a" has the value "a" in it's tags property.

{{if contains (property (module "app-a") "tags") "a"}}foo{{end}}

CLI Documentation

Complete documentation

Demo

Credits

mbt is powered by these awesome libraries

Icons made by Freepik from www.flaticon.com is licensed by CC 3.0 BY

bashiransari/mbt

mbt