PIGSTY: Postgres in Graphic STYle
Pigsty is a monitoring system that is specially designed for large scale PostgreSQL clusters. Along with a postgres cluster provisioning solution. It also shipped with a four-node VM sandbox environment based on vagrant for demonstration purpose.
- Monitoring System based on prometheus & grafana &
pg_exporter - Provisioning Solution based on ansible. Kubernetes style, scale at ease.
- HA Deployment based on patroni. Self-healing and failover in seconds
- Service Discovery based on DCS (consul / etcd), maintainence made easy.
- Offline Installataion without Internet access. fast and secure.
- Infrastructure as Code. Fully configurable and customizable.
- Based on PostgreSQL 13 and Patroni 2. Tested under CentOS 7
If you already have vagrant and virtualbox installed. These commands will just setup everything for you.
# run under pigsty home dir
make up # pull up all vagrant nodes
make ssh # setup vagrant ssh access
make init # init infrastructure and databaes clusters
sudo make dns # write static DNS record to your host (sudo required)
make mon-view # monitoring system home page (default: admin:admin) Verified version: MacOS 10.15, Vagrant 2.2.10, Virtualbox 6.1.14
Check Quick Start for more inforamtion
Pigsty provides a battery-included Monitoring System. Which is specially designed for managing large-scale PostgreSQL clusters, and consist of thousands of metrics and dozens of dashboards.
PostgreSQL cluster comes before monitoring system. That's why pigsty is shipping with a Provisioning Solution. It allows you to create, update, scale your postgres cluster in kubernetes style.
# most common database cluster management operations:
vi conf/all.yml # declare cluster status (check configuration guide for detail)
./ins-add.yml -l <host> # setup new instance / adjust instance according to config
./ins-del.yml -l <host> # remove instance on hostHere is an example base on vagrant 4-node demo. Vagrantfile define four nodes: meta , node-1 , node-2, node-3. Check Architecture Overview for more information.
Pigsty has HA Deployment support powered by Patroni 2.0.
Failover and switchover are extremely simple and fast. It can be completed in seconds without affecting any standby traffics (PG13).
One-line failover, and complete in seconds
# run as postgres @ any member of cluster `pg-test`
$ pt failover
Candidate ['pg-test-2', 'pg-test-3'] []: pg-test-3
Current cluster topology
+ Cluster: pg-test (6886641621295638555) -----+----+-----------+-----------------+
| Member | Host | Role | State | TL | Lag in MB | Tags |
+-----------+-------------+---------+---------+----+-----------+-----------------+
| pg-test-1 | 10.10.10.11 | Leader | running | 1 | | clonefrom: true |
| pg-test-2 | 10.10.10.12 | Replica | running | 1 | 0 | clonefrom: true |
| pg-test-3 | 10.10.10.13 | Replica | running | 1 | 0 | clonefrom: true |
+-----------+-------------+---------+---------+----+-----------+-----------------+
Are you sure you want to failover cluster pg-test, demoting current master pg-test-1? [y/N]: y
+ Cluster: pg-test (6886641621295638555) -----+----+-----------+-----------------+
| Member | Host | Role | State | TL | Lag in MB | Tags |
+-----------+-------------+---------+---------+----+-----------+-----------------+
| pg-test-1 | 10.10.10.11 | Replica | running | 2 | 0 | clonefrom: true |
| pg-test-2 | 10.10.10.12 | Replica | running | 2 | 0 | clonefrom: true |
| pg-test-3 | 10.10.10.13 | Leader | running | 2 | | clonefrom: true |
+-----------+-------------+---------+---------+----+-----------+-----------------+Pigsty is intergreted with Service Discovery based on DCS (consul/etcd). All service are automatically registed to DCS. Which eliminate lots of manual maintenance work. And you can check health status about all nodes and service in an intuitive way.
Consul is the only DCS that is supported (etcd will be added further). You can use consul as DNS service provider to achieve DNS based traffic routing.
Pigsty supports offline installation. It is especially useful for environment that has poor network condition.
Pigsty comes with a local Yum repo that includes all required packages and its dependencies. You can download pre-packed offline packages or make it on your own in another node that have internet or proxy access. Check Offline Installation for detail.
System Requirement
- CentOS 7 / Red Hat 7 / Oracle Linux 7
- CentOS 7.6/7.8 is highly recommened (Fully tested under minimal installtion)
Minimal setup
- Self-contained single node, singleton database
pg-meta - Minimal requirement: 2 CPU Core & 2 GB RAM
Standard setup ( TINY mode, vagrant demo)
- 4 Node, including single meta node, singleton databaes cluster
pg-metaand 3-instances database clusterpg-test - Recommend Spec: 2Core/2GB for meta controller node, 1Core/1GB for database node
Production setup (OLTP/OLAP/CRIT mode)
- 200+ nodes, 3 meta nodes , 100+ database clusters
- Verified Spec: Dell R740 / 64 Core / 400GB Mem / 3TB PCI-E SSD
Business support for pigsty is available. Contact for more detail.
- Complete version of monitoring system.
- Production deployment & operation & administration scheme
- Meta database and data dictionary
- Log collecting system and daily log summary
- Backup / Recovery plan
- Deployment assistance and trouble shooting.
- Intergration with existing system.
Read more about Business Support
Author：Vonng (fengruohang@outlook.com)