The mds-core
repo contains a deployable reference implementation for working with MDS data. It is a beta release meant for testing by cities and other entities to gather feedback and improve the product.
The Mobility Data Specification (MDS) is a project of the Open Mobility Foundation (OMF) focused on digitally managing dockless e-scooters, bicycles and carshare in public spaces.
mds-core
is...
- a reference MDS implementation usable by cities
- on-ramp for developers joining MDS ecosystem
- a tool for validating software implementations and data
mds-core
is not...
- the only implementation of MDS
- where the specification is officially defined
- a place to define local policies or performance metrics
- a cloud service that will be operated by the OMF
See the mds-core
Wiki for more details and help, including how to use it, architecture diagrams, release goals, how to help, the technical stack used, and slideshows and a video presentation.
The included code represents what is currently up and running for Los Angeles as well as new features under development. Includes the following:
- A current LADOT implementation of all MDS endpoints
- Development versions of mds-audit, mds-policy, and mds-compliance
- MDS logging (mds-logger), daily metrics (mds-daily) and Google sheet reporting app for technical compliance.
Read the CONTRIBUTING.md document for rules and guidelines on contribution, code of conduct, license, development dependencies, and release guidelines.
- MDS-Agency 0.4.0 Implementation
- MDS-Policy 0.4.0 Implementation
- MDS-Agency
/stops
PR - MDS-Audit PR
- MDS-Compliance PR
- MDS-Config
- MDS-Daily
- MDS-Metrics-Sheet
- MDS-Policy-Author
- MDS-Web-Sockets
If you haven't installed PostegreSQL and Redis you can install them with homebrew on macOS
brew install postgresql
brew install redis
Make sure they are running before you run the tests
brew services start postgresql
brew services start redis
If you encounter the following error:
FATAL: database “<user>” does not exist
The following command should fix your issue
createdb -h localhost
To run tests, you will need this:
createdb -h localhost mdstest
Then add export PG_NAME=mdstest
to your shell's environment file. (The name is not important, but you'll need to point it somehwere.)
You should have NVM already installed from the link above. The top level directory of the project has a .nvmrc
file and you should be able to run nvm install
to get the right version of Node.
Install Lerna
yarn global add lerna
Install all packages. Uses Yarn workspaces.
yarn install
Now you can work with each package
cd packages/mds-audit
yarn test
yarn start
You can also run all tests from the project root with
yarn test
This repository is a monorepo and uses Lerna for working with its packages.
Run all test suites at once
lerna run test
Run all tests suites sequentially
lerna run test --concurrency 1
Run tests for a particular package
lerna run test --scope mds-audit
Clean all dependencies
lerna run clean
Format all files
lerna run prettier
- Select the
Node.js Express Server
debug configuration - Select the file that implements the Node/Express server for a package (typically
server.ts
) in the Explorer panel - Press
F5
- Select the
Node.js: Mocha Tests
debug configuration - Select any one of the files in a package's test folder
- Press
F5
MDS can readily be provisioned to a Kubernetes capable cluster, be it a local or remote. The following steps describe how to build, deploy and operate against a local MDS cluster.
Obtain a local working copy of MDS:
% git clone https://github.com/lacuna-tech/mds-core
% cd mds-core
OSX (Linux and Windows tbd)
Install Docker Desktop:
% open https://download.docker.com/mac/stable/Docker.dmg
Start Docker-Desktop:
% open /Applications/Docker.app
Lastly, configure Kubernetes:
select the 'Preferences' option
select the 'Resources' option
apply the following minimal resource changes:
CPUs: 6
Memory: 8G
Swap: 1G
select the 'Kubernetes' option
select 'Enable Kubernetes' option
select 'Apply & Restart'
Verify:
% which kubectl
% kubectl config set-context docker-desktop
% kubectl cluster-info
In order to build and operate MDS, a number of suporting technologies are leveraged by ensuring they are installed and operational via a one-time bootstap
process:
% ./bin/mdsctl -p local bootstrap
The principle tools are: homebrew, bash-4.x+, oq, jq, yarn, nvm, helm-2.14.1, k9s, kubectx, nsc, git, gcloud and awscli. Additionally the following services are provisioned: istio and nats.
Verify:
% kubectl -n istio-system get pods
% kubectl -n nats get pods
% k9s &
Compiling and packaging MDS into a deployable form is achived as follows:
% ./bin/mdsctl build
Verify:
% docker images | grep mds*
(tbd: ?best profile?)
% ./bin/mdsctl -p processors install:mds
Verify:
% curl localhost/agency
Due to the nature of mds-core
being a highly portable Typescript project that compiles down into minified javascript for its images, rapidly development in-cluster can be quite challenging. mds-core
utilizes Okteto to enable developers to actively develop their code in-cluster.
After following the above steps to set up a local MDS cluster, you can override an existing service's deployment with these steps.
- Update
mds-core/okteto.yml
'sname
field to be set to the service you wish to replace (e.g.mds-agency
)
% curl https://get.okteto.com -sSfL | sh
- Install the
Remote - Kubernetes
VSCode extension. - Run
> Okteto Up
from the VSCode command palette.
- After the remote session opens, execute this in the new shell window:
% yarn
% cd packages/${SERVICE_NAME}
% yarn start
- This session is now safe to close, and you can reattach with the
okteto.${SERVICE_NAME}
ssh profile automatically added for you using the VSCodeRemote - SSH
package. - When you're completely done with your session, run
> Okteto Down
from the VSCode command palette, orokteto down
from terminal to revert the changes made by Okteto, and return your service to its previous deployment.
MDS operates atop the following services: Kubernetes, Istio, NATS, PostgreSQL and Redis.
(tbd)
Access the database:
% ./bin/mdsctl cli:postgresql
Access the cache:
% ./bin/mdsctl cli:redis
(tbd) Access the event stream:
% ./bin/mdsctl install:natsbox
Access the MDS cluster:
% k9s
Display the complete set of operations:
% ./bin/mdsctl
% ./bin/mdsctl uninstall:mds uninstall
To commit code, you will need the pre-commit tool, which can be installed via brew install pre-commit
. For more information, see SECURITY.md