Allow specifying the master as a zk:// uri
Opened this issue · 6 comments
It is more web scale that way.
Currently portainer uses the pure-python implementation of the Mesos framework APIs (pesos) and we're still working on bringing stable zookeeper master detection and failover to that library. The use of pesos allows us to avoid requiring libmesos for the scheduler and executor, but with the upcoming HTTP API the landscape is changing quite a bit.
Though yes, it's definitely something that we should focus on implementing.. perhaps that means switching back to libmesos on the scheduler side to gain the benefits of zookeeper master detection.
Just use kazoo and setup a watch on the (default, but configurable) /mesos znode. It shouldn't be that hard, right? :)
It's not that it's hard, it's that we've just not done it yet ;) You're more than welcome to submit a patch to pesos, or a patch to switch to libmesos, that'd be awesome!
No expert in pesos (yet), but from a 2 minute scan of the code in the github ui, PesosSchedulerDriver led me to MasterDetector, which led me here. I wanted to see how hard it would be to add, but the design is perfectly abstract to make something like this relatively straightforward.
Is it possible that pesos has added support for a ZK uri as the master_uri and you didn't realize it yet? Looks like it supports it pretty clearly to me unless I'm obviously missing something, if so then I guess yay!
Yep, pesos does indeed have some rudimentary ZK support however it's not very stable. I PRd some patches (wickman/pesos#19) around the code but admittedly didn't have the time to fully test it out, if I remember correctly there were still a few gaps in the implementation.
Let me know if you give it a try out! My expectation is that pesos will either be replaced, or refactored given the new HTTP API coming very soon.
Awesome, thanks!
Oh and the talk about MesosCon on portainer was really good. Big fan.