SEGV when executing Rancher QuickStart
mkysel opened this issue · 9 comments
mkysel commented
Describe the bug
Both 0.8.2 and 0.9.1 crash.
$ terraform apply --auto-approve
tls_private_key.global_key: Refreshing state... [id=b3c34b929a2189be69e309fd390cfeedb06139d1]
local_file.ssh_public_key_openssh: Refreshing state... [id=96fce928ff86ed690accd79aacde37bb348682ea]
local_file.ssh_private_key_pem: Refreshing state... [id=2ad081904b0d65b5b0198a7f7080b5c64b71cca4]
aws_key_pair.quickstart_key_pair: Refreshing state... [id=quickstart-rancher-20210614132641271900000001]
aws_security_group.rancher_sg_allowall: Refreshing state... [id=sg-0a5a934b9713f491a]
aws_instance.rancher_server: Refreshing state... [id=i-0bb1031c9f4b3a2bc]
module.rancher_common.rke_cluster.rancher_cluster: Refreshing state... [id=70d460ff-14cd-4fee-9363-178f5b035ed0]
module.rancher_common.local_file.kube_config_server_yaml: Refreshing state... [id=bc9fab1d082c8448b57bdc3e97040faa4cefe61a]
module.rancher_common.k8s_manifest.cert_manager_crds[0]: Refreshing state... [id=::apiextensions.k8s.io/v1::CustomResourceDefinition::certificaterequests.cert-manager.io]
module.rancher_common.k8s_manifest.cert_manager_crds[5]: Refreshing state... [id=::apiextensions.k8s.io/v1::CustomResourceDefinition::orders.acme.cert-manager.io]
module.rancher_common.k8s_manifest.cert_manager_crds[1]: Refreshing state... [id=::apiextensions.k8s.io/v1::CustomResourceDefinition::certificates.cert-manager.io]
╷
│ Warning: Empty provider configuration blocks are not required
│
│ on ../rancher-common/provider.tf line 2:
│ 2: provider "local" {
│
│ Remove the local provider block from module.rancher_common.
│
│ (and one more similar warning elsewhere)
╵
╷
│ Error: Plugin did not respond
│
│ with module.rancher_common.k8s_manifest.cert_manager_crds[3],
│ on ../rancher-common/kubernetes.tf line 8, in resource "k8s_manifest" "cert_manager_crds":
│ 8: resource "k8s_manifest" "cert_manager_crds" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).UpgradeResourceState call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.rancher_common.k8s_manifest.cert_manager_crds[4],
│ on ../rancher-common/kubernetes.tf line 8, in resource "k8s_manifest" "cert_manager_crds":
│ 8: resource "k8s_manifest" "cert_manager_crds" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).UpgradeResourceState call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.rancher_common.k8s_manifest.cert_manager_crds[0],
│ on ../rancher-common/kubernetes.tf line 8, in resource "k8s_manifest" "cert_manager_crds":
│ 8: resource "k8s_manifest" "cert_manager_crds" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.rancher_common.k8s_manifest.cert_manager_crds[5],
│ on ../rancher-common/kubernetes.tf line 8, in resource "k8s_manifest" "cert_manager_crds":
│ 8: resource "k8s_manifest" "cert_manager_crds" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.rancher_common.k8s_manifest.cert_manager_crds[2],
│ on ../rancher-common/kubernetes.tf line 8, in resource "k8s_manifest" "cert_manager_crds":
│ 8: resource "k8s_manifest" "cert_manager_crds" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).UpgradeResourceState call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.rancher_common.k8s_manifest.cert_manager_crds[1],
│ on ../rancher-common/kubernetes.tf line 8, in resource "k8s_manifest" "cert_manager_crds":
│ 8: resource "k8s_manifest" "cert_manager_crds" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more details.
╵
Stack trace from the terraform-provider-k8s_v0.9.1 plugin:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1dd0986]
goroutine 25 [running]:
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).RESTMapping.func1(0xc000d37248, 0x100df38)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:277 +0x46
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).checkAndReload.func1(0xc000392780, 0xc000d37300, 0x0, 0x0)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:179 +0x6b
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).checkAndReload(0xc000392780, 0x295f2e0, 0xc0003df4c0, 0xc000d37300, 0x0, 0x0)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:180 +0x8f
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).RESTMapping(0xc000392780, 0xc000042150, 0x14, 0xc0000ec06b, 0x18, 0xc0007360a0, 0x1, 0x1, 0xc0005286c0, 0x3, ...)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:275 +0x178
sigs.k8s.io/controller-runtime/pkg/client.(*clientCache).newResource(0xc0004abb80, 0xc000042150, 0x14, 0xc000042165, 0x2, 0xc0000ec06b, 0x18, 0xc0000ec000, 0x18, 0x0, ...)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/client_cache.go:63 +0x265
sigs.k8s.io/controller-runtime/pkg/client.(*clientCache).getResource(0xc0004abb80, 0x296d4f8, 0xc00018f288, 0x0, 0x0, 0x0)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/client_cache.go:91 +0x251
sigs.k8s.io/controller-runtime/pkg/client.(*unstructuredClient).Get(0xc000618eb8, 0x2995500, 0xc00019a1d0, 0x0, 0x0, 0xc0000ec085, 0x1b, 0x296d4f8, 0xc00018f288, 0x0, ...)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/unstructured_client.go:175 +0x152
sigs.k8s.io/controller-runtime/pkg/client.(*client).Get(0xc000618ea0, 0x2995500, 0xc00019a1d0, 0x0, 0x0, 0xc0000ec085, 0x1b, 0x296d4f8, 0xc00018f288, 0x1b, ...)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/client.go:163 +0xaf
github.com/banzaicloud/terraform-provider-k8s/k8s.resourceK8sManifestRead(0xc00020a770, 0x240d520, 0xc00054c4e0, 0xc00020a770, 0x0)
/Users/nandork/go/src/github.com/banzaicloud/terraform-provider-k8s/k8s/resource_k8s_manifest.go:237 +0x588
github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc00052e090, 0xc0000a09b0, 0x240d520, 0xc00054c4e0, 0xc00018f278, 0x0, 0x0)
/Users/nandork/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.15.0/helper/schema/resource.go:460 +0x12e
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ReadResource(0xc0005e8f50, 0x2995570, 0xc000984750, 0xc0001909c0, 0xc0005e8f50, 0xc000984750, 0xc0009a8b80)
/Users/nandork/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.15.0/internal/helper/plugin/grpc_provider.go:525 +0x3dd
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ReadResource_Handler(0x265f560, 0xc0005e8f50, 0x2995570, 0xc000984750, 0xc000190900, 0x0, 0x2995570, 0xc000984750, 0xc000a74000, 0xa15c)
/Users/nandork/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.15.0/internal/tfplugin5/tfplugin5.pb.go:3269 +0x214
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000093080, 0x29a8018, 0xc000603080, 0xc0000ae900, 0xc000618600, 0x3462690, 0x0, 0x0, 0x0)
/Users/nandork/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:1024 +0x522
google.golang.org/grpc.(*Server).handleStream(0xc000093080, 0x29a8018, 0xc000603080, 0xc0000ae900, 0x0)
/Users/nandork/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:1313 +0xd2c
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0000405f0, 0xc000093080, 0x29a8018, 0xc000603080, 0xc0000ae900)
/Users/nandork/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:722 +0xab
created by google.golang.org/grpc.(*Server).serveStreams.func1
/Users/nandork/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:720 +0xa5
Error: The terraform-provider-k8s_v0.9.1 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.
Steps to reproduce the issue:
Run the Rancher quickstart AWS
https://github.com/rancher/quickstart/tree/master/aws
Expected behavior
No SEGV.
bonifaido commented
Hi, What terraform version is this?
mkysel commented
terraform --version
Terraform v1.0.0
on darwin_amd64
dominicrico commented
@mkysel if you have a terraform.tfstate file in there, delete it. That helped me!
mjaggard commented
I'm getting this error when doing terraform destroy so of course the workaround of deleting terraform.tfstate is of no use to me.
bonifaido commented
I just tried applying some cert-manager CRDs with the Rancher QSG and had no issues, Terraform version 1.0.2 and k8s version v0.8.2.
The error comes deeply from the k8s library we use in the provider:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1dd0986]
goroutine 25 [running]:
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).RESTMapping.func1(0xc000d37248, 0x100df38)
/Users/nandork/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:277 +0x46
I wonder what could be the issue here, maybe the cluster is inaccessible for a moment, have you tried it several times?
We can upgrade the library to see if that can fix this issue.
mkysel commented
I have tried it multiple times over the course of about 2 days. Always the same result on the same machine in the same environment.
I tried it today, still SEGVs.
vbuckler commented
I have the same issue running a terraform apply having previously run a terrafrom destroy, so it's not an issue with the cluster being inaccessible. As @dominicrico says issue is only there when tfstate file are present.
The version of controller-runtime being used is a year old, might be a bug within that?
a-chernykh commented
I am getting this error when trying to switch from S3 remote to Terraform Cloud:
panic: runtime error: invalid memory address or nil pointer dereference
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x16cfbce]
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2:
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: goroutine 47 [running]:
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).RESTMapping.func1(0xc00095b300, 0x40cb68)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:277 +0x3e
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).checkAndReload.func1(0xc00059fe50, 0xc00095b3b0, 0x0, 0x0)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:179 +0x6d
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).checkAndReload(0xc00059fe50, 0x1f533c0, 0xc0009ff3c0, 0xc00095b3b0, 0x0, 0x0)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:180 +0x8c
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).RESTMapping(0xc00059fe50, 0x0, 0x0, 0xc000751dad, 0x6, 0xc000507d10, 0x1, 0x1, 0xc00074a6c0, 0x3, ...)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/apiutil/dynamicrestmapper.go:275 +0x15d
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: sigs.k8s.io/controller-runtime/pkg/client.(*clientCache).newResource(0xc0001286e0, 0x0, 0x0, 0xc000751da9, 0x2, 0xc000751dad, 0x6, 0xc000751d00, 0x6, 0x0, ...)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/client_cache.go:63 +0x277
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: sigs.k8s.io/controller-runtime/pkg/client.(*clientCache).getResource(0xc0001286e0, 0x1f65a20, 0xc0000b3690, 0x0, 0x0, 0x0)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/client_cache.go:91 +0x241
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: sigs.k8s.io/controller-runtime/pkg/client.(*unstructuredClient).Get(0xc0006bc7c8, 0x1f96d20, 0xc0000ac1f8, 0xc000751da0, 0x7, 0xc000751db5, 0x3, 0x1f65a20, 0xc0000b3690, 0x0, ...)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/unstructured_client.go:175 +0x14c
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: sigs.k8s.io/controller-runtime/pkg/client.(*client).Get(0xc0006bc7b0, 0x1f96d20, 0xc0000ac1f8, 0xc000751da0, 0x7, 0xc000751db5, 0x3, 0x1f65a20, 0xc0000b3690, 0x3, ...)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/client/client.go:163 +0xa7
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: github.com/banzaicloud/terraform-provider-k8s/k8s.resourceK8sManifestRead(0xc00057b7a0, 0x1888140, 0xc0001fbd30, 0xc00057b7a0, 0x0)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/src/github.com/banzaicloud/terraform-provider-k8s/k8s/resource_k8s_manifest.go:237 +0x568
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0000ec870, 0xc00007d180, 0x1888140, 0xc0001fbd30, 0xc0000b3688, 0x0, 0x0)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.15.0/helper/schema/resource.go:460 +0x119
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ReadResource(0xc0000b3c20, 0x1f96da0, 0xc000618480, 0xc000609860, 0xc0000b3c20, 0xc000618480, 0xc000950b78)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.15.0/internal/helper/plugin/grpc_provider.go:525 +0x3d8
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ReadResource_Handler(0x1b4ee40, 0xc0000b3c20, 0x1f96da0, 0xc000618480, 0xc000609800, 0x0, 0x1f96da0, 0xc000618480, 0xc0002c9000, 0xfb8)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.15.0/internal/tfplugin5/tfplugin5.pb.go:3269 +0x217
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: google.golang.org/grpc.(*Server).processUnaryRPC(0xc0000b7c80, 0x1fae200, 0xc000218900, 0xc0003fdc00, 0xc0006fca80, 0x2d9b490, 0x0, 0x0, 0x0)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:1024 +0x501
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: google.golang.org/grpc.(*Server).handleStream(0xc0000b7c80, 0x1fae200, 0xc000218900, 0xc0003fdc00, 0x0)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:1313 +0xd3d
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0000ac6d0, 0xc0000b7c80, 0x1fae200, 0xc000218900, 0xc0003fdc00)
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:722 +0xa1
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: created by google.golang.org/grpc.(*Server).serveStreams.func1
2022-01-10T22:32:42.767Z [DEBUG] plugin.terraform-provider-k8s_v0.8.2: /Users/nandi/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:720 +0xa1
2022-01-10T22:32:42.768Z [DEBUG] plugin: plugin process exited: path=.terraform/providers/registry.terraform.io/banzaicloud/k8s/0.8.2/linux_amd64/terraform-provider-k8s_v0.8.2 pid=1629 error="exit status 2"
a-chernykh commented
I've forked the provider and updated sigs.k8s.io/controller-runtime. It did not solve the problem but allowed me to get to the bottom of it. I realized that the Kubernetes API server was returning an Unauthorized response and I needed to update aws-auth secret to include the user which AWS token I was using. After doing this, the problem was fixed.
Here's my fork if anyone needs it: https://registry.terraform.io/providers/andreychernih/k8s/latest
-----------------------------------------------------: timestamp=2022-01-11T02:06:30.924Z
2022-01-11T02:06:30.926Z [INFO] plugin.terraform-provider-k8s_v0.9.2: 2022/01/11 02:06:30 [DEBUG] Kubernetes API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 401 Unauthorized
Content-Length: 129
Audit-Id: c5df85ea-5e7c-4b83-9a48-ea013bf142f4
Cache-Control: no-cache, private
Content-Type: application/json
Date: Tue, 11 Jan 2022 02:06:30 GMT
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}
-----------------------------------------------------: timestamp=2022-01-11T02:06:30.926Z
2022-01-11T02:06:30.927Z [INFO] plugin.terraform-provider-k8s_v0.9.2: 2022/01/11 02:06:30 [DEBUG] Received error: &errors.StatusError{ErrStatus:v1.Status{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ListMeta:v1.ListMeta{SelfLink:"", ResourceVersion:"", Continue:"", RemainingItemCount:(*int64)(nil)}, Status:"Failure", Message:"Unauthorized", Reason:"Unauthorized", Details:(*v1.StatusDetails)(nil), Code:401}}: timestamp=2022-01-11T02:06:30.927Z
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: �[0m�[1mmodule.environment.aws_route53_record.web_app: Refreshing state... [id=Z33EB3M8189PQM_app.pre.docyt.com_A]�[0m
�[0m�[1mmodule.environment.aws_sns_topic.upload_completion: Refreshing state... [id=arn:aws:sns:us-west-2:773096937500:docyt-pre-upload-completion]�[0m
�[0m�[1mmodule.environment.module.mongodb.aws_instance.mongodb: Refreshing state... [id=i-0791f4d064fcb3703]�[0m
�[0m�[1mmodule.environment.aws_acm_certificate_validation.cert: Refreshing state... [id=2021-01-10 20:54:45 +0000 UTC]�[0m
�[0m�[1mmodule.environment.aws_acm_certificate_validation.web_app_cert: Refreshing state... [id=2021-10-19 21:07:49 +0000 UTC]�[0m
panic: runtime error: invalid memory address or nil pointer dereference
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1193da5]
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2:
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: goroutine 35 [running]:
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).RESTMapping.func1()
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/client/apiutil/dynamicrestmapper.go:255 +0x45
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).checkAndReload.func1(0x28, 0x38)
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/client/apiutil/dynamicrestmapper.go:155 +0x91
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).checkAndReload(0xc000355680, {0x1fab9e0, 0xc00061a340}, 0xc00074b1b0)
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/client/apiutil/dynamicrestmapper.go:156 +0x9c
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).RESTMapping(0xc000355680, {{0xc0000da2c0, 0xc000344138}, {0xc000049b99, 0x3}}, {0xc0000e3930, 0x1, 0x1})
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/client/apiutil/dynamicrestmapper.go:253 +0x168
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: sigs.k8s.io/controller-runtime/pkg/client.(*clientCache).newResource(0xc000141290, {{0xc0000da2c0, 0x6}, {0xc0000da2c7, 0x7}, {0xc000049b99, 0x13}}, 0x0, 0x52)
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/client/client_cache.go:66 +0x28b
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: sigs.k8s.io/controller-runtime/pkg/client.(*clientCache).getResource(0xc000141290, {0x1fc89d8, 0xc000437568})
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/client/client_cache.go:102 +0x2eb
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: sigs.k8s.io/controller-runtime/pkg/client.(*unstructuredClient).Get(0xc0006a0168, {0x1fde838, 0xc0000cdec0}, {{0xc000049b80, 0x34}, {0xc000049bae, 0x137615d}}, {0x2036940, 0xc000437568})
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/client/unstructured_client.go:179 +0x1c7
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: sigs.k8s.io/controller-runtime/pkg/client.(*client).Get(0x1ce9a23, {0x1fde838, 0xc0000cdec0}, {{0xc000049b80, 0x7}, {0xc000049bae, 0x13}}, {0x2036940, 0xc000437568})
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/client/client.go:247 +0xa6
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: github.com/banzaicloud/terraform-provider-k8s/k8s.resourceK8sManifestRead({0x1fde838, 0xc0000cdec0}, 0xc0003d1e80, {0x191f7e0, 0xc00003afa0})
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /Users/andrey/github/terraform-provider-k8s/k8s/resource_k8s_manifest.go:242 +0x4ac
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xc0001b4540, {0x1fde7c8, 0xc0000af200}, 0x24, {0x191f7e0, 0xc00003afa0})
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/resource.go:347 +0x12e
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0001b4540, {0x1fde7c8, 0xc0000af200}, 0xc000627880, {0x191f7e0, 0xc00003afa0})
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/resource.go:624 +0x35b
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc000453368, {0x1fde7c8, 0xc0000af200}, 0xc0000af240)
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/grpc_provider.go:575 +0x4bd
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: github.com/hashicorp/terraform-plugin-go/tfprotov5/server.(*server).ReadResource(0xc0007e64a0, {0x1fde870, 0xc0000e0030}, 0xc0000cdb60)
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.3.0/tfprotov5/server/server.go:298 +0x1fb
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x1bfc3c0, 0xc0007e64a0}, {0x1fde870, 0xc0000e0030}, 0xc0000cdb00, 0x0)
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.3.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:344 +0x170
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: google.golang.org/grpc.(*Server).processUnaryRPC(0xc0007cca80, {0x2012578, 0xc00079ed80}, 0xc0006477a0, 0xc0007e5500, 0x2ed54b0, 0x0)
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1297 +0xccf
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: google.golang.org/grpc.(*Server).handleStream(0xc0007cca80, {0x2012578, 0xc00079ed80}, 0xc0006477a0, 0x0)
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1626 +0xa2a
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: google.golang.org/grpc.(*Server).serveStreams.func1.2()
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:941 +0x98
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: created by google.golang.org/grpc.(*Server).serveStreams.func1
2022-01-11T02:06:30.930Z [DEBUG] plugin.terraform-provider-k8s_v0.9.2: /usr/local/opt/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:939 +0x294