A service-endpoint for validating pings against mozilla-pipeline-schemas
.
See bug 1452166 for motivating background.
Start the docker container to start the local service at localhost:8000
.
The following command will fetch the latest image from dockerhub.
docker run -it mozilla/edge-validator:latest
Simply POST to the endpoint to check if a document is valid.
The testing
namespace has an example schema for validation.
$ OK_DATA="$(echo '{"payload": {"foo": true, "bar": 1, "baz": "hello world"}}')"
$ curl -X POST -H "Content-Type: application/json" -d "${OK_DATA}" localhost:8000/submit/testing/test/1
> OK
The endpoint will return 200 OK
on a successful POST.
The response will be 400 BAD
if the posted documented does not pass validation.
If the URI is malformed, the validator may return with a 404 NOT FOUND
.
The status will also include the exception that caused the error.
$ BAD_DATA="$(echo '{"payload": {"foo": null, "bar": "3", "baz": 55}}')"
$ curl -X POST -H "Content-Type: application/json" -d "${BAD_DATA}" localhost:8000/submit/testing/test/1
> BAD: ('type', '#/properties/payload/properties/foo', '#/payload/foo')
In this example, payload.foo
should be a boolean and payload.baz
should be a string.
Currently, only the first validation exception will be propagated to the user.
The exposed port can be changed through the PORT
environment variable.
It is possible to mount a set of local json-schemas by mounting a folder structure mirroring mozilla-services/mozilla-pipeline-schemas
to the container's /app/resources/schemas
directory.
$ cd mozilla-pipeline-schemas
$ docker run -v "$(pwd)"/schemas:/app/resources/schemas -it edge-validator
The generic ingestion specification provides enough context to map the ping to a schema.
The namespace distinguishes different data collection systems from each other. Telemetry is the largest consumer of the ingestion system to date. The document type differentiates messages in the ingestion pipeline. For example, the schemas of the main and crash pings share little overlap. The document version allows for versioning between documents. Finally, the document id is used to check for duplicates. This is validated in the running pipeline, but not supported here.
POST /submit/<namespace>/<doctype>/<docversion/[<docid>]
The schemas are mounted under the application directory /app/resources/schemas
with the following convention:
/schemas/<NAMESPACE>/<DOCTYPE>.<DOCVERSION>.schema.json
The following tree shows a subset of the resource directory.
/app/resources
└── schemas
├── telemetry
│ ├── anonymous
│ │ └── anonymous.4.schema.json
│ ├── core
│ │ ├── core.1.schema.json
│ │ ├── core.2.schema.json
│ │ ├── core.3.schema.json
│ │ ├── core.4.schema.json
│ │ ├── core.5.schema.json
│ │ ├── core.6.schema.json
│ │ ├── core.7.schema.json
│ │ ├── core.8.schema.json
│ │ └── core.9.schema.json
│ ├── crash
│ │ └── crash.4.schema.json
│ ├── main
│ │ └── main.4.schema.json
│ └─── ...
│ │ ├── ...
│ │ ├── ...
│ │ └── ...
└── testing
└── test
└── test.1.schema.json
The edge-validator implements the Edge Server POST request specification for Firefox Telemetry. The validator will reroute the request as a generic ingestion request.
POST /submit/<namespace>/<docid>/<appName>/<appVersion>/<appUpdateChannel>/<appBuildId>
# clone and set the working directory
$ git clone --recursive https://github.com/mozilla-services/edge-validator.git
$ cd edge-validator
# if the `--recursive` option was omitted, then update and initialize the submodule
$ git submodule update --init
# make sure that the system pip is up to date
$ pip install --user --upgrade pip
# install pipenv for managing the application environment
$ pip install --user pipenv
# bootstrap for test/report/serve
$ make sync
The docker environment is suitable for running a local service or for running any of the testing suites.
$ make shell
# Alternatively
$ docker run -it edge-validator:latest pipenv shell
If you don't require permanent changes to the engine itself, you may pull down a prebuilt docker image through
DockerHub using the mozilla/edge-validator:latest
image.
$ docker --version # ensure that docker is installed
$ make build # build the container
$ make serve # start the service on localhost:8000
The docker host automates the following bootstrap process.
pipenv
should be installed on the host system.
$ pipenv shell # enter the application environment
$ pipenv sync # update the environment
$ flask run --port 8000 # run the application
Unit tests do not require any dependencies and can be run out of the box. The sync command will copy the test resources into the application resource folder.
$ make sync
$ make test
You may also run the tests in docker in the same way as CI.
A junit.xml
file is generated in a test-reports
folder.
IMAGE=edge-validator:latest ./test_docker.sh
An integration report gives a performance report based on sampled data.
Ensure that the AWS cli is correctly configured.
$ aws s3 ls s3://telemetry-test-bucket/
Then run the report.
# Run using the local app context
$ make report
# Run using the docker host
$ EXTERNAL=1 PORT=8000 make report
The report can also be run in Docker when given the correct permissions.
$ docker run \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-it edge-validator:latest \
make report
You may also be interested in a machine consumable integration report.
$ pipenv run ./integration.py report --report-path test-reports/integration.json