Algorithm Management Toolkit (AMT)

AMT is a modern tool to apply technical and non-technical tests for an AI model.

Note: The main branch may be in an unstable or even broken state during development. Please use releases instead of the main branch.

How to contribute

See contributing docs

How to build and develop AMT

See build docs

How to run AMT

See usage docs

How to run the CLI

The check-state can be executed to see what tasks are waiting to be done.

It requires 2 parameters. The first parameter is a list of instrument urns joined by a comma without spaces. The second parameter is the location of the system card.

An example command:

./check-state urn:nl:aivt:ir:td:1.0 urn:nl:aivt:ir:iama:1.0 example/system_test_card.yaml

For developers

When running the GitHub actions locally you can use act, to do this run change the matrix in the ci.yml of the test-local job to have only python version 3.11. Then run the following command:

act -W '.github/workflows/ci.yml' -s GITHUB_TOKEN="$(gh auth token)" --artifact-server-path tmp/artifacts -e act_event.json