[Feature]: Enable to apply custom LLM Guardrails to the model
Opened this issue · 0 comments
haofeif commented
Checklist
- I've searched for similar issues and couldn't find anything matching
- I've discussed this feature request in the K8sGPT Slack and got positive feedback
Is this feature request related to a problem?
None
Problem Description
A lot of the customers mandate default LLM guardrails to be deployed as baseline responsible AI policies for all the LLM usage. For instance, bedrock has its dedicated ApplyGuardrail Independent API to apply to the input or output from the LLM, regardless of which LLM it is using. Without the LLM Guardrail, the LLM model could not be used due to company security policy.
Solution Description
Thinking of an option for K8SGPT to plug and use 3rd party Guardrail APIs (i.e. Bedrock Guardrail API)
Benefits
This will unblock the adoption for K8SGPT in organization which mandate company responsible AI policy
Potential Drawbacks
Increase the complexity of the solution
Additional Information
N/A