elasticsearch-dump/elasticsearch-dump

*** LOOKING FOR NEW ELASTICDUMP MAINTAINER ***

evantahler opened this issue · 16 comments

evantahler commented

Hello ElasticDump Users!

There often comes a time in an open-source project's lifecycle when the original maintainer needs to move on. Personally, I no longer use Elasticsearch in my day-to-day activities, and I've moved on from TaskRabbit, the wonderful company that sponsored this project in the first place. Since then, I've been maintaining this project passively for over a year, but now I find that I don't have the time to give it the focus it requires.

I would hate to see a project used by so many people rot away without active support. We've got over 2,000 GitHub stars and over 20,000 downloads a month (source). I would love to see a member of the community here step up and take over the stewardship of this project!

There are so many places Elasticdump can go in the future, and that direction is up to you! Here are a few ideas:

  • More import/export formats, like JSON, CSV, BSON, GZIP, and more!
  • Better integration with AWS (as many of the features added in the past year work with AWS authentication directly)
  • Better 'resume' features, to start where you left off should your dump become interrupted
  • Smarter parallelization
  • Better limit/search/offset tools
  • And... whatever you can think of!

I guess there are some bugs and compatibility issues to fix as well...

We've also got a fairly popular Docker image (with over 35K pulls) (stats), and distribution on a few linux package managers. I'm listing all of these to point out that if you are looking for a way to help make a difference in the Elasticsearch ecosystem, perhaps taking over stewardship of this project is for you!

If you are interested, please comment on this issue.
Thank you.

softinio commented

@evantahler I am interested. Lets discuss?

evantahler commented

Hi @softinio, and thank you for the offer!

I'd be wary of handing over the keys to someone who has yet to land a commit in the project. Is there an issue you want to tackle first or a feature you want to land to get up-to-speed in the project?

softinio commented

Good idea , but I have decided not to proceed for now as have another project I want to work on.

Thanks for getting back to me.

victorssilva commented

Hi @evantahler! Any issue or feature you want to throw in for anyone who's interested?

evantahler commented

A good starting point would be #265 and #234, which are both about how --offset work, and should be fairly simple to understand... we don't pass the offset to Elasticseaerch, and actually download all the data... just throwing it away until --offset is reached

datashaman commented

Hi there I'd be willing to give it a shot. I've got a commit or two under my belt already, and I maintain a few mildly popular projects already.

evantahler commented

@datashaman welcome aboard!

datashaman commented

Thanks!

evantahler commented

Actually, please keep this open. More than one maintainer isn't a bad thing!

datashaman commented

Roger that. :)

datashaman commented

Do you have a Slack or somewhere else that we can discuss things on?

evantahler commented

Nope, not for this project. We had a gitter a while back, but it was very underutilized. Probably the best way to chat would be on/about a github issue.

paulbjensen commented

Just wanted to say thanks for building this project, we've used it a fair bit at Starcount and it's been very helpful.

  • 👍2
rogeruiz commented

@datashaman, I may be interested in helping out too. I'll take a look at the issues @evantahler mentioned. 👍

  • 👍2
datashaman commented

Excellent, I am currently snowed under with work. I have an idea of how to speed up large indice copies (use the chunk by ID pattern from DB use), but no time to implement ATM.

ferronrsmith commented

Closing this. Always feel free to contribute 👍