Deploy an Openshift 4 cluster using UPI method (User Provisioned Infrastructure) in a disconnected scenario without a cloud provider to simulate a real baremetal environment.
The cluster is deployed on top of libvirt/kvm using Terraform and every step of the installation and configuration is automated to be able to destroy and deploy a new cluster in less than 30 minutes.
Red Hat Quay has been chosen as the registry to store the cluster images.
Warning
|
This is a demo environment and it should never be used in production. |
The provider does not currently support to create volumes with different mode
than root:root
so QEMU agent must run as priviledged:
sed -i '/^#user/s/^#//' /etc/libvirt/qemu.conf
sed -i '/^#group/s/^#//' /etc/libvirt/qemu.conf
Restart libvirt daemon:
systemctl restart libvirtd
Set the current environment:
export OCP_ENVIRONMENT="lab"
Get your pull secret from cloud.redhat.com
and copy it to the
src/openshift-install/${OCP_ENVIRONMENT}
folder (create the folder for your
environment if it does not exist):
src/openshift-install
└── ${OCP_ENVIRONMENT}
└── pull-secret.json
Deploy Openshift 4 cluster:
make
Mirror the images used during the installation to the external registry:
./output/openshift-install/${OCP_ENVIRONMENT}/mirror-release-image.sh
Go to the Quay GUI to verify that the images have been mirrored (e.g. https://registry.ocp.bmlab.int:5443).
Wait until bootstrap is ready to power on master nodes:
for i in $(seq 0 2); do virsh start "ocp-master0$i"; done
Wait until the master nodes are up and running:
./openshift-install wait-for bootstrap-complete \
--dir output/openshift-install/${OCP_ENVIRONMENT}
Finally, shutdown the bootstrap node:
virsh shutdown ocp-bootstrap
Power on worker nodes:
for i in $(seq 0 2); do virsh start "ocp-worker0$i"; done
Approve all pending CSR to allow nodes registration (run the command twice to ensure that all the certificates are signed):
export KUBECONFIG="$(pwd)/output/openshift-install/${OCP_ENVIRONMENT}/auth/kubeconfig"
oc get csr \
-o go-template='
{{range .items}}
{{if not .status}}
{{.metadata.name}}{{"\n"}}
{{end}}
{{end}}' |\
xargs oc adm certificate approve
Wait until the installation is completed:
./openshift-install wait-for install-complete \
--dir output/openshift-install/${OCP_ENVIRONMENT}
Verify the Openshift cluster is correctly created:
oc get nodes
Once Openshift 4 has been deployed in can be configured with the instructions in
day-two
folder resources.
Mirror the images for the new version to the external registry:
./output/openshift-install/${OCP_ENVIRONMENT}/mirror-release-image.sh \
"4.y.z" --apply-release-image-signature
Start the cluster upgrade to the desired version:
./output/openshift-install/${OCP_ENVIRONMENT}/update-cluster.sh "4.y.z"
Wait until the installation is completed:
$ oc adm upgrade
Cluster version is 4.y.z