/pxc-release

BOSH release of Percona Xtradb Cluster

Primary LanguageGoApache License 2.0Apache-2.0

pxc-release

Percona Xtradb Cluster release Only for limited production use

pxc-release is a BOSH release of MySQL Galera that can be used as a backing store for Cloudfoundry. The Galera Cluster Provider is Percona Xtradb Cluster. This release is intended as a drop-in replacement for cf-mysql-release.

Deploying

Deployment Topology

Galera Clustered Mysql Topology (pxc-mysql job)

The pxc-mysql BOSH job runs mysql using Galera replication, across 1 or several nodes.

The typical clustered topology is 2 proxy nodes and 3 mysql nodes running the pxc-mysql BOSH job. The proxies can be separate vms or co-located with the pxc-mysql nodes.

You can also run this topology with a single mysql node running the pxc-mysql BOSH job and a single proxy job. In this case you would have a galera cluster of size 1, which does not provide high-availability.

Database nodes

The number of mysql nodes should always be odd, with a minimum count of three, to avoid split-brain. When a failed node comes back online, it will automatically rejoin the cluster and sync data from one of the healthy nodes.

Proxy nodes

Two proxy instances are recommended. The second proxy is intended to be used in a failover capacity. You can also choose to place a load balancer in front of both proxies, or use BOSH DNS to send traffic to both proxies.

In the event the first proxy fails, the second proxy will still be able to route requests to the mysql nodes.

The proxies both will route traffic to the lowest-indexed healthy galera node, according to the galera index (not bosh index).

Traffic to the MySQL cluster is routed through one or more proxy nodes. The current proxy implementation is Switchboard. This proxy acts as an intermediary between the client and the MySQL server, providing failover between MySQL nodes. The number of nodes is configured by the proxy job instance count in the deployment manifest.

NOTE: If the number of proxy nodes is set to zero, apps will be bound to the IP address of the first MySQL node in the cluster. If that IP address should change for any reason (e.g. loss of a VM) or a proxy was subsequently added, one would need to re-bind all apps to the IP address of the new node.

For more details see the proxy documentation.

Deploying

Deploying CF with pxc-release (using the clustered topology)

Use the cf-deployment manifests with the use-pxc.yml ops file.

<a name='deploying-clustered>

Deploying pxc-release clustered

To deploy a clustered deployment, use the pxc-deployment.yml manifest and apply the use-clustered opsfile:

bosh -d <deployment> deploy --ops-file operations/use-clustered.yml pxc-deployment.yml

Deploying pxc-release standalone

To deploy a standalone deployment, use the pxc-deployment.yml manifest:

bosh -d <deployment> deploy pxc-deployment.yml

Migrating from cf-mysql-release

Requirements:

cf-mysql-release v36.12.0 or greater

Migrating CF with pxc-release

Use the cf-deployment manifests with the migrate-cf-mysql-to-pxc.yml ops file. It is advisable to take a backup first.

  • ⚠️ migrate-cf-mysql-to-pxc.yml will scale down a cluster to a single node. This is required for migration. Be sure to re-set to the appropriate number of instances when switching to use-pxc.yml subsequently.

The ops file will trigger the same migration procedure described in Using PXC release with other deployments

After migrating, use the Deploying CF with pxc-release docs for your next deploy.

Using PXC release with other deployments

  1. Make backups according to your usual backup procedure.

  2. Get the latest pxc bosh release from bosh.io

  3. Add the release to your manifest

  4. ⚠️ Scale down to 1 node and ensure the persistent disk has enough free space to double the size of the mysql data.

  5. Make the following changes to your bosh manifest:

    • Add the pxc-mysql job from pxc-release to the instance group that has the mysql job from cf-mysql-release

    • Configure the pxc-mysql job with the same credentials and property values as the mysql job

    • To run the migration:

      • Set the cf_mysql_enabled: false property on the mysql job
      • Set the pxc_enabled: true property on pxc-mysql job
      • Switch the proxies to use the proxy job from pxc-release instead of cf-mysql-release
      • Deploy using BOSH

    • To prepare for the migration, but not run it immediately:

      • Set the cf_mysql_enabled: true property on the mysql job
      • Set the pxc_enabled: false property on pxc-mysql job
      • Deploy using BOSH
      • The MySQL will run as normal with only the cf-mysql-release running
      • In order to trigger the migration, redeploy with cf_mysql_enabled: false and pxc_enabled: true

    • ⚠️ Do not enable both releases or disable both releases. Only enable one at a time.

  6. The migration is triggered by deploying with cf_mysql_enabled: false and pxc_enabled: true. The pre-start script for the pxc-mysql job in pxc-release starts both the Mariadb MySQL from the cf-mysql-release and the Percona MySQL from pxc-release. The migration dumps the MariaDB MySQL and loads that data into the Percona MySQL. This is done using pipes, so the dump is not written to disk, in order to reduce the use of disk space. The MariaDB MySQL is then stopped, leaving only the Percona MySQL running.

    • ⚠️ MySQL DB will experience downtime during the migration

  7. After the migration, you can optionally clean up your deployment:

    • The migration will make a copy of the MySQL data on the persistent disk. To reduce disk usage, you can delete the old copy of the data in /var/vcap/store/mysql after you feel comfortable in the success of your migration. Do NOT delete the new copy of the data in /var/vcap/store/pxc-mysql.
    • Deploy only the pxc-release and not the cf-mysql-release in future deployments per Deploying new deployments, to free up disk space used by the cf-mysql-release.

  8. Scale back up to the recommended 3 nodes, if desired.

Notes

  • As of pxc 0.15.x, we implemented bpm support in the pxc-mysql job. bpm puts a hard time limit on monit stop operations and will eventually SIGKILL all processes in the bpm container if mysql takes too long to shut down. When pxc-release is deployed in a Galera topology, this will cause the node to reinitialize via an SST operation. During SST a node will remove its local data directory and replace it with data provided by another member of the cluster.

Contribution Guide

The Cloud Foundry team uses GitHub and accepts contributions via pull request.

Contributor License Agreement

Follow these steps to make a contribution to any of our open source repositories:

  1. Ensure that you have completed our CLA Agreement for individuals or corporations.

  2. Set your name and email (these should match the information on your submitted CLA)

     git config --global user.name "Firstname Lastname"
 git config --global user.email "your_email@example.com"

General Workflow

  1. Fork the repository
  2. Check out master of pxc-release
  3. Create a feature branch (git checkout -b <my_new_branch>)
  4. Make changes on your branch
  5. Deploy your changes using pxc as the database for cf-deployment to your dev environment and run CF Acceptance Tests (CATS)
  6. Push to your fork (git push origin <my_new_branch>) and submit a pull request

We favor pull requests with very small, single commits with a single purpose.

Your pull request is much more likely to be accepted if it is small and focused with a clear message that conveys the intent of your change.