Upgrade issues

[cunningr]

/kind bug

Submitting for information only

What steps did you take and what happened:

I had a working CAPI cluster with v1.2.1 and openstack provider v0.6.3 and a single (test) workload cluster with the following versions:

cluster.x-k8s.io/v1beta1
controlplane.cluster.x-k8s.io/v1beta1
bootstrap.cluster.x-k8s.io/v1beta1
infrastructure.cluster.x-k8s.io/v1alpha6

Using the capi-operator I upgrade to:

v1.7.3 and openstack provider v0.10.3. After the upgrade I tried to make modifications to the workload cluster using the above apiVersions as part of a helm chart:

-> new OSMT
-> modify MD and KCP to consume new OSMT

module.cluster.helm_release.openstack_cluster: Still modifying... [id=lab-sjc2-orange, 10s elapsed]
╷
│ Error: [resource mapping not found for name: "cluster-dev1" namespace: "" from "": no matches for kind "OpenStackCluster" in version "infrastructure.cluster.x-k8s.io/v1alpha5"
│ ensure CRDs are installed first, resource mapping not found for name: "cluster-dev1-control-plane-92730156" namespace: "" from "": no matches for kind "OpenStackMachineTemplate" in version "infrastructure.cluster.x-k8s.io/v1alpha5"
│ ensure CRDs are installed first, resource mapping not found for name: "cluster-dev1-md-0-f6b31fc9" namespace: "" from "": no matches for kind "OpenStackMachineTemplate" in version "infrastructure.cluster.x-k8s.io/v1alpha5"
│ ensure CRDs are installed first]
│ 
│   with module.cluster.helm_release.openstack_cluster,
│   on modules/cluster/cluster.tf line 2, in resource "helm_release" "openstack_cluster":
│    2: resource "helm_release" "openstack_cluster" {
│ 
╵
make[1]: *** [Makefile.muon:106: terraform-apply] Error 1

There is a mix of terraform and helm trying to apply this. At this point I start experimenting and trying to recover older CRDs so the following error may not be valid:

module.cluster.helm_release.openstack_cluster: Still creating... [10s elapsed]
╷
│ Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: [ValidationError(OpenStackCluster.spec): unknown field "allowAllInClusterTraffic" in io.x-k8s.cluster.infrastructure.v1beta1.OpenStackCluster.spec, ValidationError(OpenStackCluster.spec): unknown field "cloudName" in io.x-k8s.cluster.infrastructure.v1beta1.OpenStackCluster.spec, ValidationError(OpenStackCluster.spec): unknown field "externalNetworkId" in io.x-k8s.cluster.infrastructure.v1beta1.OpenStackCluster.spec, ValidationError(OpenStackCluster.spec.identityRef): unknown field "kind" in io.x-k8s.cluster.infrastructure.v1beta1.OpenStackCluster.spec.identityRef, ValidationError(OpenStackCluster.spec.identityRef): missing required field "cloudName" in io.x-k8s.cluster.infrastructure.v1beta1.OpenStackCluster.spec.identityRef, ValidationError(OpenStackCluster.spec.managedSecurityGroups): invalid type for io.x-k8s.cluster.infrastructure.v1beta1.OpenStackCluster.spec.managedSecurityGroups: got "boolean", expected "map", ValidationError(OpenStackCluster.spec.network): unknown field "name" in io.x-k8s.cluster.infrastructure.v1beta1.OpenStackCluster.spec.network, ValidationError(OpenStackCluster.spec): unknown field "subnet" in io.x-k8s.cluster.infrastructure.v1beta1.OpenStackCluster.spec]
│ 
│   with module.cluster.helm_release.openstack_cluster,
│   on modules/cluster/cluster.tf line 2, in resource "helm_release" "openstack_cluster":
│    2: resource "helm_release" "openstack_cluster" {
│ 
╵
make[1]: *** [Makefile.muon:106: terraform-apply] Error 1

What did you expect to happen:

Some more graceful failure?

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster API Provider OpenStack version (Or git rev-parse HEAD if manually built): v0.6.3 -> v0.10.3
• Cluster-API version: v1.2.1 -> v1.7.3
• OpenStack version: Antelope
• Kubernetes version (use kubectl version): {Major:"1", Minor:"24", GitVersion:"v1.24.2", GitCommit:"f66044f4361b9f1f96f0053dd46cb7dce5e990a8", GitTreeState:"clean", BuildDate:"2022-06-15T14:14:10Z", GoVersion:"go1.18.3", Compiler:"gc", Platform:"darwin/arm64"}
• OS (e.g. from /etc/os-release):

[mdbooth]

Thanks for this, Richard.

It looks like you hit the problem that v1alpha5 is no longer served in v0.10. The detail is in the v0.10 release note: https://github.com/kubernetes-sigs/cluster-api-provider-openstack/releases/tag/v0.10.0

As mentioned in the release note, unlike v1alpha6 and v1alpha7, upgrades from v1alpha5 were never well tested. Editing the CRD to serve v1alpha5 may work for you, though, and all objects will be automatically updated to v1beta1.

Note that when we release v0.11 v1alpha5 will no longer be present at all, so this really is your last chance to do this automatically. Honestly, though, you might be better off writing new config. API changes are documented here: https://cluster-api-openstack.sigs.k8s.io/topics/crd-changes/