Orchestrator server for invocation of detectors on text generation input and output [name subject to change]
Make sure Rust and Cargo are installed. Make sure you install protobuf.
# Install system dependencies
sudo dnf install git rustup gcc perl
# Install protoc
PROTOC_VERSION=26.0
cd /tmp
curl -L -O https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-linux-x86_64.zip
unzip protoc-*.zip -d /usr/local && rm protoc-*.zip
# Install Rust tools
rustup-init -y
. ~/.bashrc # If you're on bash, so commands rustc and cargo become available.
To build and install the binary locally:
```sh
cargo install --path .
To run the server locally:
cargo run --bin fms-guardrails-orchestr8
To run tests:
cargo test
To build documenation:
cargo doc
If you wish to contribute to the project, make sure you install pre-commit hooks to have automated checks for Rust before every commit (requires pre-commit to be installed in the system).
# You can skip this step if you already have it installed.
pip install pre-commit
Once you have pre-commit installed, run the following command on the project folder to install the pre-commit hooks:
pre-commit install
- Guardrails with text generation
curl -v -H "Content-Type: application/json" --request POST --data '{"model_id": "dummy_model_id", "inputs": "dummy input"}' http://localhost:8033/api/v1/task/classification-with-text-generation
- Guardrails with streaming text generation
curl -v -H "Content-Type: application/json" --request POST --data '{"model_id": "dummy_model_id", "inputs": "dummy input"}' http://localhost:8033/api/v1/task/server-streaming-classification-with-text-generation
- Health Probe
curl -v http://localhost:8034/health
Server configuration args can also be provided through environment variables.
- For TLS, provide
TLS_KEY_PATH
andTLS_CERT_PATH
for paths to the server key and cert respectively. - For mTLS, additionally provide
TLS_CLIENT_CA_CERT_PATH
for the path to the client CA (certificate authority). - To configure log levels, adjust
RUST_LOG
todebug
,info
,warn
,error
, etc.