Is this project active?
macedo opened this issue · 23 comments
Is this project active and production ready?
I hope it's still active since I'm using it!... as for 'production ready', well, it isn't tagged with 1.0.0 yet, so I guess the answer is 'no'?
Any updates on whether this project is active or not? I'd really like to use it.
... so do I, so do I. The more I use it, the more I love it, and now I'm scared there won't be ever an upgrade (not that one is needed, but... one never knows!)
... the irony is that @philippgille goes a really long way to list in detail all the many other projects in GitHub that also implement KV-stores, their advantages (few) and disadvantages (lots), and how long ago they have been abandoned. Sadly, we have had no news here since the last commit in April 2020. I hope that @philippgille is in good health and able to read these messages, and at least make a commit or two every other month, even if it's just to update the README, so that we can continue to use the project with the confidence that there aren't any critical security issues with it.
Granted, there is no direct alternative that is both supported and has an equivalent list of features...
Well, time passes without any upgrades, and the very interesting PRs are piling up, but the good news is that @philippgille is still around, actively contributing code towards other projects, just not this one. This means that eventually he'll have some time & opportunity at taking a look at whatever pending issues have been submitted.
@philippgille Hello! Is there any news about project status?
@philippgille if you do not have the time to maintain this project and attend to all PRs, etc. please, add me as a contributor and I will be happy to help maintain it. We use it actively in production, we love it, it works, and I am willing to sacrifice some time to maintain it.
Hello everyone, original author here 👋
First of all I'm really sorry that I haven't replied here until now. And also I'm really thankful for your continued interest and for expressing it! Both with comments in this issue and with GitHub stars.
In the time when I was actively working on this I got positive feedback about it but it wasn't really used, which discouraged me a bit. But now I see it's used in several projects, which is exciting and motivating. For example @GwynethLlewelyn I see it's used in your OpenSimulator-related project, which is super interesting because I remember reading about OpenSimulator when I was active in Second Life a long time ago (programming in Linden Scripting Language is part of what inspired me to become a software engineer).
I will try to resume work on this project with the focus on the following points for the beginning:
- Check / clean up issues and PRs
- Check if the CI pipeline still works, potentially move from Travis CI to GitHub Actions or so
- Upgrade Go version in the
go.modfiles, library dependencies and Docker images (which are used during testing, like for Postgres, Redis, Consul etc.)
That's basically the gardening part, to make sure everything is up-to-date (no security vulnerabilities, packages work with latest server/service versions etc.).
Next could be some improvements / feature development, but probably no new encoding or store implementations yet, as with each one the maintenance work increases. Instead we could think about adding a context.Context parameter to all methods, or continue with the "multi" store implementation (a store with the regular gokv interface that uses multiple backing stores transparently in configurable ways like doing Set() calls to the second and subsequent backing stores asynchronously while for Get() it can iterate through the backing stores and return the first value it finds etc.)
If I don't find enough time to work on this myself, I'll try to ask for contributions specifically helping with those points. If the latter works well, a GitHub organization could be formed with multiple maintainers.
Does that sound good for everyone?
That sounds more than reasonable, for me at least :)
Moving from Travis CI to GitHub Actions is very likely a good idea; I'm really not familiar with most CI tools out there, but I found GitHub Actions to be easy enough to set up.
What a coincidence that you have been doing things in Second Life/OpenSimulator — off-topic now, but one of my goals would be to port LibreMetaverse to Go — a daunting task, worthy of a lifetime of programming! (that's why I'm currently restricting myself to just put a thin RESTful layer on top of it) And next would come OpenSimulator itself... that, at least, is a bit more modular, and I believe that one could start replacing some components here and there, which are already prepared to be stand-alone (such as the asset server).
Anyway... we'd probably also love a 'feature request' page :) (or perhaps a topic in the Discussions space) I'd love to see the (suggested)List()/GetAll() method to retrieve all keys :-) I know that's a bit beyond the purpose of the simplistic (but effective!) API, and could/should be delegated to a higher level, but the truth is that I'm completely stumped at how to implement that using the current API (granted, I'd need a few more things, namely, the ability to retrieve only a subgroup of keys, according to some simple criteria, or not-so-simple ones such as a nice regexp, but that's ok, for my purposes, I can do that in memory). And sure, I agree with your suggestion that the 4 basic primitives would be at the core of the SimpleStorage object, while additional goodies would be part of an AdvancedStorage object. That totally makes sense to me — especially because some backends (such as Redis or in-memory storage) would possibly work with am AdvanceStorage, but others might only be able to work with a SimpleStorage. Having both options sounds like an excellent solution!
@philippgille there are so many PRs waiting to be merged. Some of them are really easy to proceed with, some probably a bit more complex. But can we start moving on them?
@uded I totally agree; let's get those PRs merged, and start working from there towards the future!
🗞️ Yesterday I updated the develop branch and merged it into master and then deleted it. Having a develop branch was useful in the past before Go modules existed / became popular, where go get -u would get you the latest master and not a tagged commit. Nowadays with Go modules and people updating with @latest it gets you the latest tagged version, so if a commit on master accidentally breaks something (that wasn't noticed in a PR) it shouldn't affect anyone. This adheres to the GitHub flow instead of the Git flow and is more popular in the Go ecosystem.
I also started updating all dependencies: First from Go 1.13 to Go 1.18, then each module's dependencies. This is work in progress on the upgrade-dependencies branch. Currently I'm testing each of the modules locally, including the latest Docker image for those modules that have a server component (Postgres, Redis, etc.). When everything is up-to-date and works it's easier to migrate the CI pipeline.
Regarding the open PRs, unfortunately the deletion of develop closed many PRs automatically which targeted develop, but it should be possible to reopen them after retargeting to master. I'd prefer to do this only later though (after the dependency update and CI migration), because looking at them only few seem to be mergeable as-is.
@GwynethLlewelyn Great idea to have a "Feature requests" discussion page! I enabled the discussions feature and created one generic thread here: #107. I guess some of the existing issues (like the one about the query feature or additional storage-specific implementations) could be moved there.
🗞️ All modules are now upgraded to their latest version within the same major version (#108). Hazelcast was v0 before, so in line with go get behavior it was upgraded to v1. Few others have newer major versions which I'll upgrade to next. But with this current upgrade, and build/test.sh running all tests successfully (plus tablestorage against the real Azure because of an issue with the emulator), the migration from Travis CI to GitHub Actions could also be done now. I'll think about which one to prioritize.
I just migrated from Travis CI to GitHub Actions (#110). It was pretty straightforward and all tests are passing: https://github.com/philippgille/gokv/actions/runs/2404453799
Some open tasks are now:
- Improve GitHub Actions workflow, for example to not run tests for all modules when only a single module changed
- Upgrade some major dependencies (as I mentioned in the previous comment I only updated dependencies within the same major version so far)
- Potentially some other improvements / fixes
- Tag a new version, potentially
v0.7.0, so that all the improvements are available to projects that usegokvin a regular way (workaround until then:go getwith@master)- What I'm not sure about yet is the GitHub Release. The
v0.6.0one was a single one covering thev0.6.0tag for all storage implementations and it introduced Go modules. With the Go modules now existing independently, and each one having their own tag (i.e. Redis implementation getting theredis/v0.7.0tag), I'm not sure if there should just be a singlev0.7.0release, mentioning all changes sincev0.6.0in the various modules, or should there be one release per module? With that many modules that seems like a lot of overhead. Forv0.7.0a single combined release might still make sense, given that at least the Go version in thego.modfiles changed for all of them. But in the future, when only a single module changes and we bump it tov0.8.0for example, there will inevitably have to be module-specific releases. Should theRELEASES.mdfile then be split as well?
- What I'm not sure about yet is the GitHub Release. The
- What I'm not sure about yet is the GitHub Release.
Speaking strictly for myself, I'd rather prefer a single release for all modules & packages of gokv — even if they are technically separate subprojects. But that's just the way I think about releases :-)
Granted, if completely separate and autonomous packages are being updated on their own, and, more importantly, if they're being used directly by any other project that does not use gokv at all... assuming that there might exist such a forced example... then, aye, I'd be fine with having those packages being 'special' and getting their own, independent and separate version numbers.
This is true for some extraordinarily complex Go projects that use lots of 'general-purpose' modules, some of which are so useful that they are actively used by completely different packages, which do not use the 'parent' project at all. I'd say that this is something that must be decided on a case-by-case basis.
For instance, a Redis module designed to work for gokv is very likely not useful for anything else, so, even if it's a separate module, I'd expect it to have the same version and GitHub release number as the 'parent' project, gokv. On the other hand, imagine that you find it useful to create an extension to the strings library that does some cool parsing of key values to pretty-print it, and, naturally enough, it becomes its own package (and module). By itself, such a package is stand-alone, even if it's located under the modules tree for gokv. While personally I would rather consider having a general-purpose package outside the gokv tree, there are plenty of examples of project owners on GitHub that think differently (possibly because they have set some permissions and configurations for moderators/PRs etc. at the 'parent' project, or do the CI at the parent level for all submodules, and don't wish to replicate everything on all other projects), and those 'sub-projects', ought, indeed, to have their own release version (since they might effectively be released independently of the 'parent' project).
I hope that my comments make any sense to you...
@philippgille can you please review #127 ?
/me *bumps* the thread, too! 😄
@philippgille can you please review #127 ?
Thank you, I'll have a look on Tuesday
to avoid major break and support backward compatibility, added WithContext functions for get/set/delete #136
feel free to comment and review this is a draft
In addition to the ⬆️ linked ones, many more PRs have been opened and merged since then. There's very little left on the list that I want to include in the next version, v0.7.0. I'm tracking the progress here, where someone asked about a new tag/release: #159
With all the ongoing progress, as well as collaboration from contributors, I'm closing this issue now. Thank you all for raising your voices and bringing this project into my focus again.
If the issue should arise again in the future, where I become unresponsive to issues, comments or PRs, don't hesitate to create a new discussion or issue! 🙇♂️
New release: v0.7.0
@philippgille thank you for keep doing amazing work!