idea: Local Provider Discoverability with Zeroconfiguration
Opened this issue · 1 comments
Problem Statement
Users of Jan want to start chatting immediately with absolutely no setup on their end.
Feature Idea
Implement ZeroconfAI!
https://github.com/jperrello/Zeroconf-AI
Technical details:
Jan does an mDNS lookup for _zeroconfai._tcp.local. and the results describe OpenAI compatible endpoints for any providers that announce themselves on the local area network.
Disclaimer: I am the main developer for zeroconfAI and I would love to have Jan be the first app to use this so I can start building a much larger ecosystem. This is not a commercial effort, it is my Master's Project in Baskin Engineering at UCSC.
We added support for ZeroConf AI to Jan in a roundabout way. We created a Python script that listens for all local service announcements and runs a local proxy server that is OpenAI compatible. Ideally, Jan would be able to listen for these announcements itself. But after that, the way the servers are communicated with is exactly the same as any other OpenAI compatible server. No API keys are needed anywhere.
I'm the research advisor the Zeroconf Al project at UC Santa Cruz. We have a pretty detailed design fiction (showing multiple use cases for the ecosystem here): https://github.com/jperrello/Zeroconf-AI/blob/main/fiction/design_fiction.md
In the same way that someone in your house has probably taken care of setting up wifi for everyone else in the house, we imagine that same local sysadmin might setup Zeroconf Al services. Apps should be able to discovery AI services on the LAN in the same way they do with printers -- usually no passwords, joining the wifi is all you need.