Questions about the state of mysos within twitter?

[chaoranxie]

hello @xujyan

Not sure what the best medium to post these questions, whether it is github issue or one of the following mailing list. let me know
dev@cotton.incubator.apache.org
dev-subscribe@mysos.incubator.apache.org

I have some general questions about the state of mysos/cotton within twitter. My guess is that each of the features currently available within twitter will be slowly open sourced and make generic so that different implementation can be contributed by the community. I am curious what they look like at twitter right now

I am curious about how twitter current using mysos to do failover, it seems like it is using the reparent.sh as you mentioned here that without gtid "some files need to be copied out of band by tools such as MHA", does the version running inside twitter already have integration with MHA? or is ther other mechanism that bring the bin log to new master.

Also i see that there is the concept of BackupStore, does twitter current using HDFS to backup the files? i see some sample code related to it here but not sure if it is just an idea or twitter already have one implementation of it.

Thanks again for open sourcing cotton, we look forward to more news

[xujyan]

Hey! Sorry about the confusion as we were transitioning code/docs into the new home: https://github.com/apache/incubator-cotton

I have now updated this repo's code and README to point to the new repo. You should use dev-subscribe@cotton.incubator.apache.org and I have just updated the README page with the new links.

w.r.t to the failover issue, it's not solved in the codebase and at Twitter we are going to leverage 5.6 GTID for this and therefore don't have plans to make this work on pre-5.6. The currently deployed code doesn't support failover and our initial use cases don't need it. We'd be happy to accept contributions for it though.

Regarding BackupStore and more generally "gradually open sourcing more internal features", the truth is that we don't have more internal features that have yet to be open sourced. We merely have our custom implementations of BackupStore and Installer, etc. that handle our internal backup workflow and internal MySQL packages. Given how other organizations manage backups and package installation, implementations of BackupStore and Installer can be very simple and a reference implementation may not be necessary. And yes we store the backups on HDFS we pull them down by using hadoop commands directly. What's your workflow?

I have updated https://github.com/apache/incubator-cotton with details about mailing list and JIRA, etc. Please subscribe and post questions over there! Thanks for your interest!

Admittedly documentation is terribly lacking currently and we'll improve on this.