Welcome to HCE's GraphQL API template documentation
This contains the API templates to perform a chaos experiment in an automated way.
-
Installed HCE: You should have an HCE installed from where you can prepare and run experiments from API calls.
-
Prepare Workflow: We can run a workflow using APIs in any automated pipeline but for that, we need to pre-create a workflow with the right tunables and attributes from HCE. This step will involve:
- Select the right set of experiments you want to include in the workflow
- Provide desired tunables in the experiment. At any point in time, you can change the tunables and save it - this won't impact the overall API calls, infact this is correct method to update the tunables of the pre-created experiment for new runs.
-
Install 'jq': Make sure you have
jq
installed in your system if not, please install it as it is used in some APIs to filter path. -
Derive Tunables: To derive tunables for preparing API commands check here.
-
Download 'hce-api' binary:
-
Follow the below-mentioned steps to install 'hce-api' binary. It will be used to prepare the API commands based on the provided tunables.
-
Download
hce-api
binary using this command, and replace<OS>
and<ACRH>
placeholder with your operating system (supportslinux
anddarwin
) and system arch (supportsamd64
andarm64
).wget https://github.com/uditgaurav/hce-api-template/releases/download/0.1.0/hce-api-<OS>-<ARCH> -O hce-api chmod +x hce-api
If you have don't have
wget
then you can also usecurl
command:curl -L https://github.com/uditgaurav/hce-api-template/releases/download/0.1.0/hce-api-<OS>-<ARCH> -o hce-api chmod +x hce-api
-
- Derive the tunables for API calls. Login to your HCE dashboard.
- On the HCE dashboard click on "Settings" and switch to the "Access Key" tab.
You will get this screen:
- Click on "Create Access Key" if you have lost the older one
For Project ID:
- On the HCE dashboard click on "Project" (as shown in point-1 on the image) and copy the "Project ID". You can also get the project ID from the HCE URL at this place.
Checkout this screen:
For Workflow ID:
- Click on "three dots" on the workflow and Navigate to the "View Manifest" option. You will get a screen like this:
It will give you the workflow id for the target workflow.
This contains the API to trigger the Chaos Experiment.
ACCESS_KEY
ACCESS_ID
PROJECT_ID
WORKFLOW_ID
HCE_ENDPOINT
The above tunables are mandatory to provide. You need to replace it in the given API call.
- Before getting started checkout the pre-requisite section and follow the steps to download
hce-api
binary.
Non-Interactive Mode:
- Provide the values in the flags given below to get the API command in a file.
./hce-api generate --api launch-experiment --hce-endpoint=<HCE-ENDPOINT> \
--project-id <PROJECT-ID> --workflow-id <WORKFLOW-ID> \
--access-key <ACCESS-KEY> --access-id <ACCESS-ID> --file-name <FILE-NAME>
Example:
./hce-api generate --api launch-experiment --hce-endpoint=http://ae1a8f465611b4c07bbbc2e7d669f533-1139615104.us-east-2.elb.amazonaws.com:9091/ --project-id abceb5f4-4268-4467-9818-ad6e3b6bfd78 --workflow-id f4581780-efaf-4155-956e-6c379f24394b --access-key nEdGNDDrTFHyCnl --access-id adminNCWQu --file-name hce-api.sh
The default value for --file-name
is hce-api.sh
, all other variables are mandatory.
Interactive Mode:
- Run the downloaded
hce-api
binary as shown below. It will not trigger the chaos but will prepare the launch command in a file:
$ ./hce-api
Provide the index number to create a file with the API command from the given options.
For Example to create an API that launches an experiment give 1 as input.
[1] API to Launch Chaos Experiment
[2] API to Monitor Chaos Experiment
[3] API to Validate Resilience Score
Select from the given options: 1
Provide the HCE endpoint: http://ae1a8f465611b4c07bbbc2e7d669f533-1139615104.us-east-2.elb.amazonaws.com:9091/
Provide the Project ID: abceb5f4-4268-4467-9818-ad6e3b6bfd78
Provide the Workflow ID: f4581780-efaf-4155-956e-6c379f24394b
Provide the HCE Access Key: nEdGNDDrTFHyCnl
Provide the HCE Access ID: adminNCWQu
Provide the File Name for API [Default is hce-api.sh]: hce-api.sh
Output:
The file containing the API command is created successfully
- Check out the file
hce-api.sh
and get the launch command. To run the API executebash hce-api.sh
(replace file name for custom names).
This contains the API to monitor the Chaos Experiment that is this API will help us to wait for the workflow completion.
-
ACCESS_KEY
-
ACCESS_ID
-
PROJECT_ID
-
WORKFLOW_ID
-
HCE_ENDPOINT
-
Before getting started checkout the pre-requisite section and follow the steps to download
hce-api
binary.
Non-Interactive Mode:
- Provide the values in the flags given below to get the API command in a file.
./hce-api generate --api monitor-experiment --hce-endpoint=<HCE-ENDPOINT> \
--project-id <PROJECT-ID> --workflow-id <WORKFLOW-ID> \
--access-key <ACCESS-KEY> --access-id <ACCESS-ID> --file-name <FILE-NAME>
Example:
./hce-api generate --api monitor-experiment --hce-endpoint=http://ae1a8f465611b4c07bbbc2e7d669f533-1139615104.us-east-2.elb.amazonaws.com:9091/ --project-id abceb5f4-4268-4467-9818-ad6e3b6bfd78 --workflow-id f4581780-efaf-4155-956e-6c379f24394b --access-key nEdGNDDrTFHyCnl --access-id adminNCWQu --file-name hce-api.sh
The default value for --file-name
is hce-api.sh
, all other variables are mandatory.
Interactive Mode:
- Run the downloaded
hce-api
binary as shown below. It will not trigger the command but will prepare the monitor command in a given file:
$ ./hce-api
Provide the index number to create a file with the API command from the given options.
For Example to create an API that launches an experiment give 1 as input.
[1] API to Launch Chaos Experiment
[2] API to Monitor Chaos Experiment
[3] API to Validate Resilience Score
Select from the given options: 2
Provide the HCE endpoint: http://ae1a8f465611b4c07bbbc2e7d669f533-1139615104.us-east-2.elb.amazonaws.com:9091/
Provide the Project ID: abceb5f4-4268-4467-9818-ad6e3b6bfd78
Provide the Workflow ID: f4581780-efaf-4155-956e-6c379f24394b
Provide the HCE Access Key: nEdGNDDrTFHyCnl
Provide the HCE Access ID: adminNCWQu
Provide the File Name for API [Default is hce-api.sh]: hce-api.sh
Output:
The file containing the API command is created successfully
- Check out the file
hce-api.sh
and get the chaos monitor command. To run the API runbash hce-api.sh
(replace file name for custom names).
- In this sample script we will wait for the workflow completion with a delay of 2 seconds and 150 retries, you can adjust these values based on total chaos duration.
#!/bin/bash
set -e
# Tunables provide your values
DELAY=2
RETRY=150
fileName="hce-api.sh"
# Script execution logic
START=0
for (( i=$START; i<=$RETRY; i++ )); do
echo
res=$(sh $fileName)
echo
if [ "$res" == "Succeeded" ]; then
echo "Experiment completed, CurrentState: $res"
exit 0
fi
sleep $DELAY
echo "Waiting for experiment completion... CurrentState: $res, retries left: $(($RETRY-$i))"
done
echo "[Error]: Timeout the workflows is not completed with DELAY: $DELAY and RETRY: $RETRY, CurrentState: $res"
exit 1
This contains the API to get the resilience score for a workflow run and validate against the expected probe success percentage.
-
ACCESS_KEY
-
ACCESS_ID
-
PROJECT_ID
-
WORKFLOW_ID
-
HCE_ENDPOINT
-
Before getting started checkout the pre-requisite section and follow the steps to download
hce-api
binary.
Non-Interactive Mode:
- Provide the values in the flags given below to get the API command in a file.
./hce-api generate --api validate-resilience-score --hce-endpoint=<HCE-ENDPOINT> \
--project-id <PROJECT-ID> --workflow-id <WORKFLOW-ID> \
--access-key <ACCESS-KEY> --access-id <ACCESS-ID> --file-name <FILE-NAME>
Example:
./hce-api generate --api validate-resilience-score --hce-endpoint=http://ae1a8f465611b4c07bbbc2e7d669f533-1139615104.us-east-2.elb.amazonaws.com:9091/ --project-id abceb5f4-4268-4467-9818-ad6e3b6bfd78 --workflow-id f4581780-efaf-4155-956e-6c379f24394b --access-key nEdGNDDrTFHyCnl --access-id adminNCWQu --file-name hce-api.sh
The default value for --file-name
is hce-api.sh
, all other variables are mandatory.
Interactive Mode:
- Run the downloaded
hce-api
binary as shown below. It will not trigger the command but will prepare the monitor command in a given file:
$ ./hce-api
Provide the index number to create a file with the API command from the given options.
For Example to create an API that launches an experiment give 1 as input.
[1] API to Launch Chaos Experiment
[2] API to Monitor Chaos Experiment
[3] API to Validate Resilience Score
Select from the given options: 2
Provide the HCE endpoint: http://ae1a8f465611b4c07bbbc2e7d669f533-1139615104.us-east-2.elb.amazonaws.com:9091/
Provide the Project ID: abceb5f4-4268-4467-9818-ad6e3b6bfd78
Provide the Workflow ID: f4581780-efaf-4155-956e-6c379f24394b
Provide the HCE Access Key: nEdGNDDrTFHyCnl
Provide the HCE Access ID: adminNCWQu
Provide the File Name for API [Default is hce-api.sh]: hce-api.sh
Output:
The file containing the API command is created successfully
- Check out the file
hce-api.sh
and get the validate-resilience-score command. To run the API runbash hce-api.sh
(replace file name for custom names).
- In this sample script you will get the probe success percentage for the last workflow run, you can make use of it to compare it with the expected probe success percentage.
#!/bin/bash
set -e
# Tunables provide your values
expectedProbeSuccessPercentage=100
fileName="hce-api.sh"
# Script execution logic
res=$(bash $fileName)
res=$(echo "$res" | tr -d '"')
echo
if [ "$res" != "$expectedProbeSuccessPercentage" ]; then
echo "The probe success percentage is: $res, expected probe success percentage: $expectedProbeSuccessPercentage"
exit 1
fi
echo "The probe success percentage is equal to the expected probe success percentage"
exit 0