Performing a full fork?
Opened this issue · 79 comments
In light of the history of this project and its maintainers, and now the (news-to-me) GPL violations, combined with the virulently toxic attitudes of the "community" around Emby (the "stop whining" crowd) - should we do a full fork? Basically, planning to do what Libresonic did when Subsonic went closed-source?
As mentioned in #25 I don't have any experience with C# development, but I know a few people who do and might be willing to get involved, and I can help with build infrastructure/hosting/etc. as that's my specialty. Frankly I don't care that "99% of users won't care", I care, and will ONLY use OSS software, which is why I even went with Emby in the first place. And I care a lot about keeping it that way for everyone else, since Emby is the only reasonable and OSS alternative to Plex. Plus this gives us the flexibility to start adding useful features (for me, LDAP user authentication support and the ability to choose between Air order and DVD order for TV without manually hacking metadata, things with the Emby core devs have ignored for years at this point).
Some short-term goals could be:
- Rename to something else.
Openby? - Opening up the build process and ensuring full GPL compatibility for all parts of the app.
- Proper packaging for Debian and related distros - no more patching, let's integrate it right into the fork and build compliant packages.
- Removal of all callbacks/links to the main Emby site from inside the server.
Some longer-term goals could be:
- Rebuild the Android app with the new name and fully OSS (Fdroid rather than Google Play?)
- Remove all "premium" components or make them fully available.
- Add additional features like LDAP, TV order switching/choice, etc. and any other features we might think of.
Any thoughts or interest from those here? Do you think it's worthwhile? Could we get enough devs to keep up?
@nvllsvm @dcrdev and anyone else.
Edit 2018-08-10: My fork is at https://github.com/joshuaboniface/Emby and should build a Debian package right now.
This is a subject I've thought long and hard about over the last few months and just the way Emby is going at the moment and the attitude of it's fans and developers towards the GPL, make me want to participate in something like this.
I think if we could get enough skilled people on board, then it's worth exploring but only then. It's one thing to fork the server component, but then the apps need developing from scratch as they have never been open source; you know the moment a fork appears, the Emby team will block it within their own apps.
But I think to be more specific, immediate priorities:
- Remove binary blobs from the repository and either remove references to those libraries or re implement them.
- Ensure the build process is as open as can be.
What I can bring to the table:
- Extensive database knowledge ANSI SQL / MSSQL / MySQl (Data Engineering is sort of what I do)
- Some .NET skills (albeit very very rusty now)
- Any sys admin duties (Linux mind you)
- RPM Packaging Skills (Sponsored RPMFusion Packager)
Also in regards to a name - I was thinking something like EmbéLibre lol
I think if we could get enough skilled people on board, then it's worth exploring but only then.
So that's 2 of us, I know @nvllsvm mentioned he does this for a friend in another issue, but I've also got a friend or two with C# experience who might be willing to help occasionally.
It's one thing to fork the server component, but then the apps need developing from scratch as they have never been open source; you know the moment a fork appears, the Emby team will block it within their own apps.
That's a concern of mine as well - given their attitude in general I'd be wary of them taking "retaliatory" action against the fork. It looks like the Android app (https://github.com/nvllsvm/emby-android) is indeed GPL though, which is a start - I'm not sure whether you guys are Android or iOS but I'm Android so that's fine by me to start with.
But I think to be more specific, immediate priorities:
Remove binary blobs from the repository and either remove references to those libraries or re implement them.
Ensure the build process is as open as can be.
I like these priority-wise, and think they're easy enough to get done as soon as we have a bit of a community. I've already got a working Debian build setup for the emby-server component (see my PR), also "forked" from their source packages, that can hopefully be used as a template for RPM as well.
Also in regards to a name - I was thinking something like EmbéLibre lol
I like that, but accent aigu might cause trouble ;-)
There's also Streama (https://github.com/dularion/streama). I'm now debating throwing my and my friend's weight behind it instead of dealing with Emby. It's still immature though moving fast and needs more work (it didn't have half the features the last time I checked on it), but it's a good start, and written in Groovy instead of C#. The only breaking issue for me is the requirement to have a theMovieDB.org API key to use it.
I did have a look at that a while back and it's certainly a valiant effort, but for me it lacks something critical - music support.
Also I know it's written in Groovy, but it's still Java and to me that's a bit off putting; I have very mixed feelings about Java. Whereas despite C# and .NET Core coming from Microsoft, I feel are generally well designed.
Welp, looks like we need to hard fork. Emby depends upon proprietary blobs else the software won't compile.
https://github.com/MediaBrowser/Emby/issues/3075#issuecomment-371021228
Finally got around to starting a side project I've been thinking about for well over a year.
In a few hours of hacking, I managed to hack together:
- a script to transcode and segment a video file into small chunks
- an HTTP service hosting the chunks, HLS manifests, and hls.js demo UI
Feel free to look at the caffeine-fueld wip. Did I mention it's hacked together?
https://github.com/nvllsvm/maelstrom/tree/dev
Anyway - seems to work so far :)
@nvllsvm That looks great! On the one hand, building a full replacement streaming server is a big undertaking, but on the other, having another option, written in Python (yay) and without a decade of .NET cruft is amazing! I'm definitely keeping an eye on it and will see if I can help! ;-)
Hi,
I would be very happy to help anywhere I can. Most of my experience is in front end web dev although I have done some simple things in C, C++ and C# in the past.
I have also got a reasonable amount of linux experience (about 10 years but never in a professional environment). I've only been using Open Source for a while and would love to give something back if I can. I use emby-unlocked a lot so thought this would be a good place to start. Please let me know if I can help.
Not sure how serious this discussion this but I have a couple of thoughts:
-
Is the intent to start over completely (as in a completely new project) or just a hard fork (ala OpenOffice vs. LibreOffice)? @joshuaboniface had a good thought of not dealing with a decade of .NET cruft and that's very appealing but I am not sure how feasible it is to start over and build a complete replacement.
-
I think the new project should focus explicitly on server and the web client (and the underlying API). I realize it may be subjective but I feel building the clients will be easier, if you are using the same API as the web client. Plus we could look at using Xamarin, so there isn't as much effort required to create an app for all the various platforms.
-
As far as developers go, there are a bunch of other projects on GitHub that we should explore and see if they would be interested in collaborating. Obviously, everyone has different goals but even if you get a couple people interested, that would be a win.
Here are a few projects that seem like a good place to start:
https://github.com/nmaier/simpleDLNA
https://github.com/MediaPortal/MediaPortal-2 (this one might be difficult, since it's already a mature application but this is the closest to what Emby is)
https://github.com/TwitchBronBron/PlumMediaCenterSharp
https://github.com/einsteinx2/WaveBox
Edit: A couple more relevant projects:
Thanks for the feedback everyone!
I think @SamAtwell 's point #1 is the big rub - we're hitting the "POSIX filesystem" problem: we can't attract any users until we have a feature-complete system (for a limited feature set, but it must be relatively close to what Streama has now if not what Emby/Plex have), but getting there is a non-trivial amount of work. I'd be more open to doing this for the reasons I mention, but it would need several solid developers (I'm not one) to dedicate to it. @nvllsvm's proof of concept looks really nice as a starting point though!
Given what we have today though, and especially with @Jab2870's offer maybe just hard-forking emby-server is a good plan. We can take @THEYsuxx's suggestion of decompiling the non-compliant DLLs, and just go from there. This is a lot less work right now, but would we have momentum long-term? And is it worth all of our effort to support a "new Emby"? It would still be the same codebase which is pretty gross. It's a hard problem.
For #2 I agree completely (again, as a non-developer) - client apps are "easy" once there's a scalable API in the backend. I've never heard of any of the projects in #3 so I can review some of them to see where they're at.
I'm contemplating something else at the moment... bear with me:
Whilst I have ended my issue on their GitHub page due to them having cleaned up a few things and there no longer being a technical GPL Violation. I still believe that the way they are handling things goes against the spirit of the GPL and the fact that one cannot build it themselves is absurd even if they are claiming those releases to be a "re-licensed version" , they are not highlighting that fact to end users.
So what I'm thinking of at the moment is forking it, writing some python to patch the csproj files dynamically and then having some shell scripts merge any changes from upstream. I could then set up builds for Fedora, CentOS/RHEL and OpenSUSE via copr - anyone else would be free to chime in for Debian/Ubuntu/Docker.
Then in regards to the propriety blobs, they could be reverse engineered but purely to get the class definitions and any methods would just throw not implemented exceptions. So in effect as a starting point we would have an entirely free version of Emby and that would not be a direct threat to them e.g. we might get away with still using their apps, but they'd have to be purchased on a single device basis due to lack of premiere; that would also continue to bring in an income for Emby.
Finally the pièce de résistance...
As you may have noticed Emby doesn't really have that many contributors from outside, a large part of this I suspect is down to their CLA. The proposed fork would apply no such restriction - hopefully this may begin to attract interest from developers.
At which point it would hopefully have enough traction to warrant the creation of new apps, a re-brand and the re-implementation/re-imagining of some of those premiere features.
Thoughts?
Also not ruling out a completely new solution - I too would like something that isn't dependent on netcore as it's a pain to manage a build system on Fedora/RHEL. I like the sound of something in C++ or Rust maybe.
But realistically that's going to take a lot of man-power from the get-go and would likely take quite a lot of time before it comes to fruition. But very open to the possibility...
Also not ruling out a completely new solution - I too would like something that isn't dependent on netcore as it's a pain to manage a build system on Fedora/RHEL. I like the sound of something in C++ or Rust maybe.
Would using something like FAKE or Cake help with that?
The one benefit to using C# (and .NET) is you get to re-use a big portion of your code for building client apps through Xamarin & Mono. IMO that's a huge advantage compared to other platforms, though I suppose you can do the same JS with React Native, Ionic, etc.
My main concern is that is can be compiled on Linux, sooooo mono support is a must 😉
I have started work on modifying Emby to build under netcore / scripting the automation to merge changes from upstream.
Have 75% of the projects building, a couple I need to iron out the dependencies for.
https://github.com/dcrdev/Emby
Also need to work out what's supported in netcore in terms of unit testing.
There's a portion of code missing from their repo that would allow you to build a fully working version under .net core. That is the server application itself - in the repo only exists the mono server application and the windows one.
I'm re implementing it to the best of my abilities at the moment, but looks like there's also some missing constructors in the 'Emby.ServerImplementations' project to support this. That and the fact every single javascript file (not 3rd party) is minified, really gives me the impression they don't welcome contributions.
So whilst I'm probably going to be able to get this working, my dream of automating it has gone swiftly down the drain - I've had to make significant changes. I've also stripped out Windows support entirely because I can't test it, also have stripped out Mono as that will become redundant anyway. Anyone like to chip in?
@dcrdev - Look around in the other MediaBrowser repos. Pretty sure the minified JavaScript files exist as their original forms in other repos. I know Emby.ApiClient.Javascript contains some of the original Javascript files, but not sure if it has them all.
Little taste of what's coming:
https://gist.github.com/dcrdev/9fd1652ebeb2e5701161882439d09765
I'm excited!
Looking good @dcrdev!
Waiting for this 😀
Looking good.
I've rewritten the apphost library now and have it up and running, just 3 things to go before I push the missing piece to the repository:
- Either ask Luke to update the NuGet package for MediaBrowser.Naming so that it targets .net standard or core, or pull in Emby.Naming into the solution directly.
- Removal of proprietary binaries as described here: dcrdev/Emby#2
- One final ambiguous "Object reference not set to an instance of an object” exception being thrown in the logs which I need to identify and squash.
I'm quite shocked at the extent to which what Emby is publishing in their git repo differs from what's coming out of their releases. For instance:
- .NET Core App Host application isn't published at all.
- The project files and NuGet setup are in no way set up for .NET Core.
- There are hard dependencies on things that do not target .NET Core.
It's so much more than missing build scripts as we had previously thought!
Also I discovered that ThirdParty/emby/Emby.MediaEncoding.dll is not open source and that's not even part of premiere, it's integral to the functionality of the core server.
I've been following this for a little while now, got interested in Emby since it was advertised as open and I run on an arm based platform w/ internal/non-standard decode/encode HW.
Just a crazy thought on the "new solution" topic above. I've dabbled in Kodi on occasion, how bad would it be to make a server version of it?
As far as I remember, Plex started as a server version of Kodi.
Also, I have ended up forking the Emby repo, I am not happy with the current state of affairs with Emby and so would like to try and move away to a fully open source fork. I have taken the last fully open source commit of Emby and gotten it building as a netstandard20 project.
I have pulled in the dependencies Emby provides too as I don't want to rely on them going forward, I do need to make attribution and licensing clearer however as some of these components are MIT licensed rather than GPL. I still need to pull in and reproduce the build system for the front end JavaScript and HTML.
If interested you can check out the fork at https://github.com/benpye/Emby , as I have included some other non GPL projects from Emby (MIT licensed) I do need to make sure the licensing is clearer.
Looks like the Emby team has completely fucked over open source fans - they've closed up critical components in the latest release.
https://github.com/MediaBrowser/Emby/issues/3075#issuecomment-406841199
It looks like this has been rectified.
@voxadam - License is still ambiguous, hopefully just an oversight MediaBrowser/Emby.Common#1
I don't disagree; I was only commenting on the most recent issue.
OK, I hard-forked the main Emby repo and deleted all commits after 3.4.1.14: https://github.com/joshuaboniface/Emby
This gets us to the stable state we were at before all the fuckery with 3.5 and the splits.
Over the next few days I'm going to integrate both the Emby Unlocked config as well as my own Debian build stuff into this repo and test that it builds successfully. I don't recommend making any issues/comments/PRs to my repo just yet in case I need to do some major refactoring, but know that it's there.
@berrnd I see in the main Emby issue about all this, you mention you're running your own fork. To avoid polluting or referencing the main project thread, I'm pinging you here.
I'd be willing to just help contribute to yours instead of trying to run start my own fork; what sort of modifications do you make to the main source? I'd much rather pool resources on this rather than make competing forks!
My changes are nothing that fits all - modified some screens (without taking care of localization), integrated a preconfigured portable Kodi installation to make streaming for friends easier, added a lot statistic things (because I love charts and numbers and stuff :D) - so really just personal customizations that are beyond the possibilty of own CSS and plugins.
I think I can't invest enough time to help maintaining a real open fork of Emby. The code base is really huge (and undocumented). In general, I think Luke does a good job with improving Emby, creating new features, etc. The only thing I can't understand is how he handles the "openness". Especially the thing with replacing the Emby.Common parts with DLLs and saying that the parts are now in its own repo, which is in fact not the same. Maybe I should open a separate issue to clarify this part...
I've already brought it up elsewhere, but would a snap package be something of interest? Deals with packaging and building for a bunch of distributions, and would deal with having to maintain a repository for updates.
Effectively with GitHub + snapcraft.io you'd have all the code/builds hosting taken care of.
Docker is equaly interesting imho if we're talking about packaging. But there's a lot of Emby Dockerfiles floating about, so makeing this should be very trivial.
So it looks like, due to the Mono build failures @dcrdev has noticed, I can't get 3.4.x.x to build. I guess I should have read that whole issue 😆
I'm going to fork the 3.3.1.X branch instead. This is the one I'm using for now anyways, and I care a lot about getting this to build on my own infrastructure as a prerequisite to making any changes to the project.
I've grabbed the source packages from https://download.opensuse.org/repositories/home:/emby/Debian_9.0/ and will put those packages into a central repo here (probably just the Emby repo on the right tag).
@jkaberg @ABotelho23 I'm definitely willing to do both, once I get the 3.3.1.X version up in the repo please feel free to submit PRs for the alternate packaging format.
I've never actually packaged a snap myself, but I am fully willing to take on that burden. My good friend @tbelway is as enthusiastic about making Emby right as myself, and would help me out.
As far as I can tell, as long as building isn't hell, it looks simple enough.
Looks like I solved most of the build issues (as far as I can tell) on the 3.4.x branch; I get an apparently-working Debian package on the other end, so assuming Snap, Docker et al work similarly it should be pretty trivial. I'm working on getting embymagick and libembysqlite3 (ulgh why did they do that!?) integrated into the package now as well just to keep all the core server components together, which from my understanding of those two formats is pretty important as well.
Windows compatibility maybe?
Have you decided if you're going to create your own repo?
Ideally packaging should be it's own set of issues on GitHub.
I'm just kinda subscribing to this thread here. I recently installed Emby and then learned of their GPL violations and such when I looked into contributing some of my own HTML, CSS, & JS code changes. So I'm no longer confident in the state of the project, and I definitely don't want to contribute to the MediaBrowser team on this one. Normally I'd have no problem with paid features in open-source projects, but obviously this goes beyond just the paywall & nagware.
Looks like there's 2 promising forks of the project here (dcrdev's and joshuaboniface's) but neither is quite ready for a newbie to just pick up and start using as is? I've personally done no C# development, but I'll be keeping an eye on both of these forks and doing what I can to contribute.
Windows compatibility maybe?
Could explain it, but on the Linux side I found that embymagick seems unnecessary with the full repo build, and I was able to replace the libembysqlite3 with normal sqlite3 and from my initial tests everything seems to be working fine.
So, if you want to build a Debian package for 3.4.1.18, my repo should "just work" - it only requires mono-devel 5.18 (the latest from the Xamarin Debian repository).
Have you decided if you're going to create your own repo?
At the moment just the "fork" one, but I'll see how it goes over the near future. I'm not going to change the name just yet or anything, unless forced to.
Ideally packaging should be it's own set of issues on GitHub.
I'd rather keep them in one repo along with the source personally, I prefer the simplicity of that. And Debian basically requires this sort of structure for its packaging - I'm really coming from that side of things ;-)
@mijofa Welcome! I personally can't help much with mine (I'm just a sysadmin who dabbles in Python but really loves Emby), but if you want to make a PR I'd be happy to test it out.
@dcrdev I see you repo hasn't had anything done to it since April or so, and is on the 3.3.1 branch. I'd be happy to merge any updates you have that would help with 3.4.1 from there, or merge my changes in to yours to keep a central point. Let me know what you think!
@joshuaboniface so embymagick and libembysqlite3 are not actually emby specific libraries, they are sqlite3 and imagemagick and have different names only to prevent incompatibilities between the required versions of those libraries and the versions shipped as part of the various distro repositories. Basically in essence they were created to prevent the Emby guys having to debug issues against lots of differing versions of those libraries across users.
Why are you still trying to go down the mono route? That's a a dead end - upstream aren't developing for mono anymore - if something breaks , it's likely broken for good.
To be honest I'd like to put in some time into my fork, but haven't for two reasons:
- Started a new job a few months back and haven't had the energy to.
- I'm slightly concerned about the upstream licensing situation and what that means for me. It's hardly well defined which parts of upstream are licensed under the GPL and which are not - least of all because all those components sit in the same repo.
@dcrdev Seems like a other great reason to package a snap to me!
So .NET would effectively be the way forward, just done.. correctly?
Yes, so I've talked about this quite extensively in the past. All current upstream releases are now built against .NET Core and .NET Standard - all development is focused on those toolkits. The trouble is neither the project definitions, nor the launcher used as an entryy point into the EmbyServer application is published in their repo. They wont tell me or anyone else why that is.
So those things need to be created from scratch - if you look at my repo, I have done all the work already on recreating those project definition files and infact the whole thing builds under .NET Standard on Linux. I've even included GNU make tooling to make this process easier on Linux.
Then comes the entry point launcher into the server . upstream has only publically released the code for the full .NET SDK i.e. Windows, the OS X and Mono launchers none of which are fully compatible with .NET Core. Again I have done the work on this and have fully re-created it by inferring from the upstream binary what it's doing, it works, it runs and as far as I know has no issues. But you'll note this is not in my repo and is infact explicitly excluded via my .gitignore. This is because it's a fine line between reverse engineering and re-implementing and given how fuzzy everything else is over there, I'm hesitant to publish it. It's not a complicated piece of code - maybe a couple of hundred lines if that; still though.
And to be honest here - where is this going to go anyway without a decisive effort on the part of the community. Upstream don't care about this clearly, I'd much rather someone come along and start something new that everyone can happily contribute to and not have to worry about this fucking nonsense from a bunch of developers who think they're god gift to the fucking universe.
On the subject of snap packages - dunno, aren't snap packages more targeted at gui applications? Why snap, why not flatpak - at least that's supported everywhere; snap doesn't support selinux so that rules out the rhel derived distros and last time I checked it wasn't in the official arch repos either; just the aur.
What's wrong with docker and don't we already have docker images?
Sounds to me like you're cloning behavior, which is similar to the UNIX/Linux relationship. I don't think you'd run into any trouble. I don't blame you for the hesitation though.
I wouldn't say snaps are designed for GUI applications. The part that's REALLY nice is particularly snapcraft, which can build snap releases directly from a GitHub repo automatically, and publish them on snapcraft.io. This project could basically live entirely on GitHub without the need for servers to host binaries, while still supporting the package being grabbed from snapcraft.io and such. The snappy package manager also auto-updates snaps by default, so users would always be on latest stable by default.
I'm fairly certain every major distro uses SELinux these days, not just RHEL derivatives.
I 100% support Docker, but snaps have without a doubt a lower barrier of entry.
@dcrdev Fair points - I'll give some of my thoughts on them.
so embymagick and libembysqlite3 are not actually emby specific libraries, they are sqlite3 and imagemagick and have different names only to prevent incompatibilities between the required versions of those libraries and the versions shipped as part of the various distro repositories. Basically in essence they were created to prevent the Emby guys having to debug issues against lots of differing versions of those libraries across users.
That makes sense - it makes packaging more of a pain but I see their reasoning. Still it seems to work fine on Linux without them, at least on Debian Stable that is. If anyone can comment on other distros as well that would be helpful. TBH I don't care a whole lot about packging it for Windows or OSX myself (I'm 100% Linux) but of course a maintainer of that would be welcome.
Why are you still trying to go down the mono route? That's a a dead end - upstream aren't developing for mono anymore - if something breaks , it's likely broken for good.
Yes, so I've talked about this quite extensively in the past. All current upstream releases are now built against .NET Core and .NET Standard - all development is focused on those toolkits. The trouble is neither the project definitions, nor the launcher used as an entryy point into the EmbyServer application is published in their repo. They wont tell me or anyone else why that is.
So those things need to be created from scratch - if you look at my repo, I have done all the work already on recreating those project definition files and infact the whole thing builds under .NET Standard on Linux. I've even included GNU make tooling to make this process easier on Linux.
Then comes the entry point launcher into the server . upstream has only publically released the code for the full .NET SDK i.e. Windows, the OS X and Mono launchers none of which are fully compatible with .NET Core.
Well this is part of why I'm hard-forking the 3.4.1.x release which still works fine with Mono 5.14. I'm not well-versed in .NET Core; I see there is a Debian package for it so I suppose there's no issue, but as far as I'm concerned as long as Mono works, there's little reason to ditch it, and it does work today.
To note, I'm hard-forking entirely to avoid upstream's BS - whatever upstream does from this point forward is their business, but I'm not very interested in continuing to backport our fixes into their constantly- and arbitrarily-changing codebase. That includes stuff like changing to .NET Core. I definitely don't plan to introduce such breaking changes into my repo for what to me is zero benefit, that's for sure!
To be honest I'd like to put in some time into my fork, but haven't for two reasons:
Started a new job a few months back and haven't had the energy to.
Definitely can't blame you and didn't intend to criticize :) Just wanted to get on the same page about the status.
I'm slightly concerned about the upstream licensing situation and what that means for me. It's hardly well defined which parts of upstream are licensed under the GPL and which are not - least of all because all those components sit in the same repo.
IANAL, but wouldn't these components still be under the GPL even without their sources, since they were all released and packaged together?
So those things need to be created from scratch - if you look at my repo, I have done all the work already on recreating those project definition files and infact the whole thing builds under .NET Standard on Linux. I've even included GNU make tooling to make this process easier on Linux.
Again I have done the work on this and have fully re-created it by inferring from the upstream binary what it's doing, it works, it runs and as far as I know has no issues. But you'll note this is not in my repo and is infact explicitly excluded via my .gitignore. This is because it's a fine line between reverse engineering and re-implementing and given how fuzzy everything else is over there, I'm hesitant to publish it. It's not a complicated piece of code - maybe a couple of hundred lines if that; still though.
I'd love to integrate these changes; as mentioned above from my knowledge of GPL and @ABotelho23's comment there, we should be in a good state since the binaries were released as GPL code and we're replicating functionality, but if we have extensive doubts it might be worth reaching out to the FSF to see where we stand license-wise.
And to be honest here - where is this going to go anyway without a decisive effort on the part of the community. Upstream don't care about this clearly, I'd much rather someone come along and start something new that everyone can happily contribute to and not have to worry about this fucking nonsense from a bunch of developers who think they're god gift to the fucking universe.
Well, this is the rub. I agree completely - I'd love to support an alternative (and preferably Python-based ;-)) that doesn't have 10+ years of .NET cruft and the stain of these developers' attitudes, but AFAIK one does not exist right now in a usable state. So for the time being, Emby is still our best bet for a usable web-based video platform. And I think a bit of effort now to get a stable Emby with some useful minor features is a good plan. Whether this fork lasts a year or not, or even gets any new minor features, will remain to be seen, but for today I don't see a much better alternative. I'm going to begin implementing LDAP auth in my fork one way or another since I'm tired of waiting for it to come to the gratis version, and also to help familiarize myself with C#.
On the subject of snap packages - dunno, aren't snap packages more targeted at gui applications? Why snap, why not flatpak - at least that's supported everywhere; snap doesn't support selinux so that rules out the rhel derived distros and last time I checked it wasn't in the official arch repos either; just the aur.
What's wrong with docker and don't we already have docker images?
(@ABotelho23) I 100% support Docker, but snaps have without a doubt a lower barrier of entry.
Why not all 3? Once the source builds packaging it should be the easy part. I'm already planning to get RPM support in addition to Debian, and Arch should be a simple drop-in as well. More packaging formats is always better IMO!
Well this is part of why I'm hard-forking the 3.4.1.x release which still works fine with Mono 5.14. I'm not well-versed in .NET Core; I see there is a Debian package for it so I suppose there's no issue, but as far as I'm concerned as long as Mono works, there's little reason to ditch it, and it does work today.
I think the point here is that mono isn't supported, while .NET Core is. So eventually it would have to be switched over anyway. We wouldn't wanna rely on obsolete software, especially when it is often internet facing (I currently host my Emby on the internet with an nginx reverse-proxy!)
Why not all 3? Once the source builds packaging it should be the easy part. I'm already planning to get RPM support in addition to Debian, and Arch should be a simple drop-in as well. More packaging formats is always better IMO!
For me the concern is the hosting of the packages; the current official Emby solution is a) unsigned packages, and b) manually updated, not via a repo
Someone would have to host a repo? That's money that needs to be spent to do that. The GitHub + snapcraft.io setup requires no infrastructure/costs nothing, and would be built automatically when stable releases are performed.
I think the point here is that mono isn't supported, while .NET Core is. So eventually it would have to be switched over anyway. We wouldn't wanna rely on obsolete software, especially when it is often internet facing (I currently host my Emby on the internet with an nginx reverse-proxy!)
Oh so you mean that Mono as a project is going away and .NET Core is all that will be left? I haven't been following that at all but if that's the case moving is indeed a good idea.
Someone would have to host a repo? That's money that needs to be spent to do that. The GitHub + snapcraft.io setup requires no infrastructure/costs nothing, and would be built automatically when stable releases are performed.
I'd be happy to host a package repo for the Emby fork to support things other than Snaps. I make extensive use of the Debian package so I need that infrastructure for myself anyways; I could always make it public and support a few more distros as well. Or just encourage everyone to build it themselves from Source - as long as we make that easy.
@joshuaboniface no criticism taken lol
Mono vs .net core? I'd say there's every reason to switch - Mono is an Linux abstraction layer for native win32 calls, it's significantly slower than the dotnet sdk and requires mono to be installed or packaged with the target executable, dotnet does not. The standard sdk vs the framework does not require special knowlege, one is merely a subset of the other.
The thing about the binaries though is that they aren't licensed under the gpl or we would have the source code for them. Luke has already confirmed that these are licensed under a non specific proprietary model. This where things get hazy because according to him we can still distribute them as part of our own builds, but we have nothing but to confirm that other than a comment on github.
In regards to getting the FSF involved, I'd be up for that - if I were able to get a clear and satisfactory answer to the question of whether I'd be legally safe in continuing what I'm doing. Then I'd be happy to push my code and then things can continue from there. If anyone would like to assist in drafting an email (there's so much to cover) , that would be a start.
I'd be happy to host a package repo for the Emby fork to support things other than Snaps. I make extensive use of the Debian package so I need that infrastructure for myself anyways; I could always make it public and support a few more distros as well. Or just encourage everyone to build it themselves from Source - as long as we make that easy.
All for it! As long as the infrastructure is there, I don't see why not of course.
Also, I meant to mention, was LDAP support not added to Emby not too long ago? I can see it as a plugin, but is the plugin proprietary or is the implementation not very good? Setting up users is the biggest pain for me setting up a new server (between family and friends I've got like 10 users across Linux users, Samba users, Emby users, etc.)
@dcrdev I'd be happy to draft something up over the next few days. And that does make total sense; I've opened an issue to switch to it on my repo. Do you know if the workflow is roughly similar to what's being done here? If so it should be a drop-in.
@ABotelho23 It was added as an extension that is AFAICT both proprietary and is explicitly limited to Premium subscribers. That's my exact issue with it too, I've got close to a dozen users and every other service I run auths against a central backend; Emby is the last outlier.
@joshuaboniface the resultant build will be structured in the same way, so you wont have to make too many changes to your packaging definition, but you'll obviously have to call a different compiler and reference the new entry point launcher:
See my makefile:
https://github.com/dcrdev/Emby/blob/master/Makefile ,
Thanks @dcrdev , looks like I'll have to do some more fighting with it (the build commands seem to exit with errors seemingly at random) but that's a good starting point to figure this out!
Ah, I see, it looks like there are massive changes to some of the csproj files to support this, which is why I can't get it to work.
@joshuaboniface yes that's what I meant by package definitions above - I've made those changes in my repo: https://github.com/dcrdev/Emby/blob/master/MediaBrowser.Model/MediaBrowser.Model.csproj
Finally just caught up reading this thread! Leaving this as a place holder for now.
It was added as an extension that is AFAICT both proprietary and is explicitly limited to Premium subscribers.
But wouldn't that be the best way for all premium feature in emby? Instead of closing down the source code and build process so nobody can remove the license checks, just make it into a plugin. They could even ship it with their super proprietary binaries if they want. I personally don't mind paying for some features, I think gitlab for example does a great job with their business model. But I'm really dissastified with the move from OSS to closed source software, especially when they don't seem to be able to provide stuff like arm64 packages and there is no way to build the project yourself.
@Kiina Absolutely, and if they hadn't been trying all the scummy stuff regarding nag screens, proprietary licensing attempts, and outright hostility to the free software part of their community, I probably would have just been another happy Premium user personally. But all of that adds up. Besides, their implementation from what I've seen is half-assed (it's an "import from LDAP" not "auth against LDAP") and about 3 years too late. Maybe if Mr Lead Dev had spent time properly implementing the feature instead of constantly trying to obfuscate his code and write anti-features, it wouldn't have been a big deal, but after everything else it is. Auth is a basic part of every web application - why is my choice of where my user passwords are managed a proprietary and premium-locked feature?
While I haven't done much of anything on my fork yet (mostly poking around to see how to implement it myself) I want to have the option available in case the whole Emby project goes closed-source like Subsonic did. If anyone following this thread wants to PR against it though I'll accept happily! I've also got an issue about changing the name waiting for more community feedback!
What is more feasible? Bringing streama to reasonable feature parity with Emby or building a self-sustaining fork from Emby? I have to think the former. I'm non-technical and just observing, but streama has a community, active development, committed dev, and MIT license. That compares to building a community from scratch with lots of emby's legacy baggage. This emby fork saga has been going on for too long and there seems to be some "low hanging fruit" for talented devs to pick off at streama that can go a long way to creating wider appeal: transcoding, music, dlna. There has already been work done towards transcoding that someone could pick up and finish: streamaserver/streama#440
Anyhow, just my perspective. I've been using Emby for a while now, but if I'm going to put time into a community, I'd prefer to make a clean break from Emby and streama checks off enough boxes for me to think that the effort wouldn't be wasted.
I've been vaguely watching Streama development for a little while and would love to see the Chromecast functionality actually get developed for that. Sadly I personally don't know Java and am not in any position to help contribute to that project directly. :(
I'm currently looking into what it would take for me to get this functionality with Python, but I would much rather use an existing project than write something myself
@vfrex, streama is written in JAVA tho, its crappy.
It always fucks up in the end etc, wrong java versions etc.
Better do a fork of emby and redo everything wrong with it.
Agreed hate to be that guy but Java is a piece of sh** that should die, it's a syntactical shitsh** that enforces a built in garbage collector that frankly creates more overhead than it helps eliminate.
Ironically I've had to start doing some Java/Scala stuff at work recently and have hated every goddamn minute of it. In comparison C# is a beautiful language - it's everything Java should have been, even though I am far from a fan of Microsoft.
The way I see it:
- Java = Sh**
- C# = Strongly typed language, pretty decent benchmarks for a high level language. Smallish open source developer pool outside enterprise applications.
- Python = Lower learning curve, lots of open source developers and community, highly mature. Syntactically I personally prefer C# - but that's personal preference. But the BIG thing here is speed and Python is fu**ing slow compared to the competition.
Then you have C, C++ and Rust but hobbyist developers are few and far between - well maybe not Rust but that's still quite new.
Anyway - are we going to continue with this quite frankly pointless discussion if no one else is willing to step up an contribute some code? Otherwise might as well close this down...
What is more feasible? Bringing streama to reasonable feature parity with Emby or building a self-sustaining fork from Emby? I have to think the former.
So I hadn't checked up on Streama in a while, and it does look to have progressed a bit from where it was when I last checked. But TBH it's still got some "drop-in replacement" dealbreakers, especially requiring a TMDB API key, as well as needing to import every file manually. Simply put, it's nowhere near feature parity with Emby and I say that knowing full well that Emby is missing some pretty critical features.
I'm not sure what the right answer to this is - if we (meaning myself and any other non-devs who can't contribute much to these projects) have to wait for a long time for Stream to be ready, then forking Emby today makes more sense - we don't have to do anything to it to make it work or build on a technical level (licensing issue aside) and can make some small, incremental improvements as needed. Long-term a replacement would definitely be worth the effort, however is Streama that replacement? And if not, what is?
I'm more focused on a fork because of the former reasons rather than the latter - I'm worried about 6 months from now when Mr Lead Dev decides to close down the whole repo and take the project closed source, rather than the hypothetical 2-years-from-now ideal solution of a full feature-complete replacement. Of course others may disagree with my opinion here and that's fine, if Streama becomes viable sooner than all the better.
I do agree though that this discussion isn't very fruitful at this point. We've got a hard fork at my repo and @dcrdev is working on the licensing issues in his fork (and sorry I haven't had much of a chance to contact the FSF yet - though in the US it seems APIs can be copyrighted so we might be screwed there either way, but that's a discussion for a separate issue).
Reopening. Emby core has gone proprietary.
https://github.com/MediaBrowser/Emby/issues/3479#issuecomment-444985456
Welp, I saw that one coming. I haven't checked up on @dcrdev's work recently, but mine stagnated as I havent had much time to learn. But I'll throw my weight there. "Libreby" here we come!
Well this totally blows. I wish I had more time to help with this but I've now started a new #dayjob so I don't really have enough time to devote to this alone.
If you want to use my stuff at https://github.com/benpye/Emby go ahead. It builds but I actually kinda regret the commit I took to base my work off, Emby had already started only publishing minified JavaScript and CSS. To be honest, the web frontend code is Emby could be better anyway, I wanted to try and improve the code but I really struggled. I got half way through undoing their minification - I did this by:
- Taking the first minified commit - I assumed this had no functional changes to the commit before - appears to hold
- Taking the last minified commit
- Running each through a code prettifier, I don't recall the exact tool!
- Taking a diff
- Manually applying the diff to the last non-minified commit
The three remaining files to apply this to are syncsettings.js, tvshows.js and videoosd.js . In addition to this all the libraries used are minified! We need to go and find out what commit was used at the time for each library and grab them from upstream, this should be relatively easy if a little tiresome. Emby has no JS build system, it would be nice to use NPM or something to grab deps.
In my fork everything but the web front-end is still open source though this comes at the cost of being a few releases behind the last open source release (3.5?). If anyone has any questions or specific difficulties I will try and help but my time to devote to this is limited! As it stands my repo should build using .NET Core and appears to run just fine.
Good luck all.
EDIT:
Few more thoughts on the possibility of a from-scratch replacement. I can see this being perhaps a better way forward. As shown above getting FFMPEG to transcode video and getting HLS playback to work isn't insanely difficult. Perhaps starting anew would be better - we could leave the crufty web UI behind, maybe use TypeScript + Angular for the front end and personally I would suggest Go on the backend but Python is great too. I like Go because deploying a Go application is trivial, it's easy to consume C libraries and you get small native binaries avoiding the packaging complexity of lots of other languages.
If starting from scratch we could ensure that the web API is a strong contract, the front end and back end should be more or less completely seperate. It would be nice if the web app could be a PWA too, it would avoid the need to worry about native mobile applications for each platform. A good API would hopefully enable more third party clients than Emby has ended up with, and hopefully we could expose config etc in the public API, something that Emby seemed to be lacking?
90% of the emby is FFMPEG, the rest is just shimmer and glimmering stuff to make it look good.
I vote for a complete rebuild and not a fork, that way one can find a good streaming backend that already are opensource and does the heavy lifting.
In my opinion Python would be a more natural choice for anything that runs on a linux/unix server, having to depend on mono to run could and has created problems. The cleaner one could get the code the better, python is readable easy to use and easy to speed up using c and c++ if needed.
The only reason i started to use Emby was because that was the close's thing I could get that ran smoothly on both linux and BSD, so my hope is that if you guys go for a fork/rebuild you make something that runs on linux or bsd.
I am not a big fan of MS or it's followers so I do not develop things to run on the MS platform at all. If you choose to go with a python version I will drop in and help as best as I can.
Hate to be that guy, but has anyone thought about go?
It's fast, statically typed, has loads of quality libraries and provides static builds for every platform available.
Damn, shame to see emby finally closing up.
Guess it's really time to I stop just talking about an alternative...
Out of curiosity, what are your use cases?
I personally run a server for myself and a few others, but nothing massive.
Whilst python isn't the fastest, I'd lean towards it simply for how easy it is to write, and also for the existing modules.
Personally I'd want to write a series of APIs, as opposed to a whole web product. That way you could write a nice simple front end to start, then make it more feature complete later.
I'm interested in rust, so I'd eventually I might look at replacing the backend with that - or at least the stuff that grabs metadata.
I've opened up an issue on my fork regarding where to go from here and how we're going to acheive it. I'd appreciate input from anyone who has made their own fork or who has opinions on how to proceed from here.
I've been working on an alternative for a while now, it's written in NodeJS and React.
Been already using it myself for over a year now.
https://github.com/OwenRay/Remote-MediaServer
Contributors are welcome:)
right now I've got:
- TV/Movie indexing via TMDB
- Videos browsable/searchable/filterable
- Supports all modern browsers
- srt/ass/webvtt Subtitle support
- HLS and h264 mp4 on-the-fly remuxing or transcoding
- End-to-end encrypted sharing
- and lots of other tiny things
@joshuaboniface if you believe your fork to be the most up to date and is building, would you be interested in an arch AUR entry for some sort of "emby-server-forkJB"?
90% of the emby is FFMPEG, the rest is just shimmer and glimmering stuff to make it look good.
I vote for a complete rebuild and not a fork, that way one can find a good streaming backend that already are opensource and does the heavy lifting.
In my opinion Python would be a more natural choice for anything that runs on a linux/unix server, having to depend on mono to run could and has created problems. The cleaner one could get the code the better, python is readable easy to use and easy to speed up using c and c++ if needed.
The only reason i started to use Emby was because that was the close's thing I could get that ran smoothly on both linux and BSD, so my hope is that if you guys go for a fork/rebuild you make something that runs on linux or bsd.
I am not a big fan of MS or it's followers so I do not develop things to run on the MS platform at all. If you choose to go with a python version I will drop in and help as best as I can.
Damn, shame to see emby finally closing up.
Guess it's really time to I stop just talking about an alternative...Out of curiosity, what are your use cases?
I personally run a server for myself and a few others, but nothing massive.
Whilst python isn't the fastest, I'd lean towards it simply for how easy it is to write, and also for the existing modules.Personally I'd want to write a series of APIs, as opposed to a whole web product. That way you could write a nice simple front end to start, then make it more feature complete later.
I'm interested in rust, so I'd eventually I might look at replacing the backend with that - or at least the stuff that grabs metadata.
Just started a project to encapsulate these ideas: https://github.com/djotaku/TransStream
Dang...so it finally happened. I suspected it would happen at some point but I was hoping I would be wrong.
As far as next steps go, I hope people consider how big of an investment it is to build a brand new project. Just building a new project alone won't be enough. You need a thriving community if it is to successful. That being said, the best decision long-term is probably to build something new and not carryover any of the baggage or the cruft. It's just going to be a lot of hard work to get there.
@bobberb It builds, but it's quite old and the other guys have done a bunch of work to port it to the newer better Mono. But, I'm more in favour of a new project long-term. (see joshuaboniface/Emby#11)
@djotaku Agreed completely on almost every point. My usecase is the same, a 11+TB media library that I want to stream on my various devices (Chromecast, Phone, Web) and give access to various friends and family. I picked Emby a few years ago precisely because it was GPL'd and as good as Plex, which everyone was clamoring to use but I disliked due to preferring Free Software.
As I outline in the issue I mention above, I have a plan to write a modular backend media server system in Python with @anthonylavado, and then just let front-ends use that. FFMPEG is indeed the heaviest lifter, but at least to me, the "shimmer" is really important - Metadata management, and a good UI to browse with on multiple devices. There's a couple really great ones out there, tons of up-and-comign projects to provide shimmer, but it seems every one is limited by not having a universal backend, which is something I want to change.
I still think keeping Emby around is a good idea, even if there's never another commit, because it works right now, so I'll spend some time making sure that's all set before proceeding with a new project, and we can let the community decide which is more valuable (i.e. if C# coders pop out of the woodwork haha!)
@joshuaboniface Do you have a repo for it yet? I basically JUST learned about this today (even though going through the comments it looks like it's been in process since April (at least)). So my repo is really just a readme with my intentions. I'm fine with you and @anthonylavado being co-owners of the repo since we have aligned goals. It wouldn't be stomping on any of my stuff as I haven't done anything today other than play with FFMPEG a bit to understand how it works for streaming. Alternatively, if you prefer to have your own repo, please let me know when it's up and I'll try and be an active contributor to yours.
Your use case's are exactly mine, and I think my mediaserver already does exactly what you have in mind (albeit a bit rough around the edges at times).
I don't want to come over as pushy; It's just that, it would be a shame if we'd all go off start something new/different just... because.
Eitherway, some of the features I've build really aligns with the vision:
- backend is JSON:API, the data is very flexibly queryable in a standardized way.
- As it stands now: supports desktop and mobile browsers and chromecast (subtitles and all)
- Supports secure sharing of files in a torrent-like fashion.
@OwenRay OK, I had only briefly looked it over. My plan would be to do a solid backend in Python (I hate how fast NodeJS moves, as a sysadmin) that supports arbitrary media so frontends could be written in ehatever language to interface with it. But I'd be happy to ibcubate multiple projects together, no need to compete but help each other build a set of great media server tools.
@OwenRay I think as soon as @joshuaboniface and I get a chance to look over RMS we should be able to comment better on it.
We’re also planning on writing, I guess you’d call it a set of specifications/use cases, so we can make sure we’re covering everything the right way.
For example, I’m really really interested in making it as modular as possible. If someone wanted to add audio later on, it should be something they can enable, but they don’t have to if they don’t want to.
Ok I agree, I tried setting it up modularly, especially later additions are very loosely coupled.
I'm open to changes for improved extensibility.
Eitherway it goes; I'd love to help out on the "specifications/use cases"