Clarification on Frontend and Backend Compatibility with Multiple Models

[bhupesh-sf]

I have a few questions regarding frontend and backend compatibility:

1. Frontend-Backend Compatibility: Does any frontend work with any backend? I noticed that most backends are designed to work with GGUF models, but this project seems to support various backends, including vLLM, Mistral.rs, etc. If I select one of these backends, will it be compatible with frontends like Open WebUI and other similar platforms?
2. Closed-Source Frontend Support: Currently, I have a closed-source frontend that is only compatible with GGUF models. If I choose to use this project, is it possible to expose a backend that my closed-source frontend can consume, allowing it to support multiple types of backends?

I’d appreciate any guidance or clarification on these points!

[av]

Thanks for trying out Harbor!

1. It's only mentioned in the docs implicitly, but the support is not complete (nor it could be, to be fair)

The policy is to support everything for the main default UI and default Backend. I.e. - integrate all UIs with the default backend and all UIs with the default backend. Anything else is considered extra and provided on the best effort basis (contributions are very welcomed!)

This is not to discourage you in any way - plenty of services combinations work out of the box, the best way to check is to lookup the x files to see if there's an intersection.

2. All the Harbor services that expose the APIs can be used with external clients. See docs on the harbor url / harbor qr commands

[bhupesh-sf]

Thanks for the answer, it does clarifies 👍🏻

[av]

Awesome, feel free to reopen if needed