Can't use third party providers with a backend?
Closed this issue · 9 comments
Hi,
I was trying to use this with https://github.com/atlassian/terraform-provider-artifactory/ - I built it myself and placed it in a directory under my terraform_source
dir called terraform.d/plugins/linux_amd64
I'm using the S3 backend with a role_arn
I was able to plan/apply with the terraform-resource, but it fails at the implicit get stage when trying to terraform init
:
fetching ljfranklin/terraform-resource@sha256:d63ba8bb3262392adfa3e68dad9b94584bcb4ec74095392f51a3a71543afa33a
c9b1b535fdd9 [==========================================] 2.7MiB/2.7MiB
45bf92bbfa9b [========================================] 12.1MiB/12.1MiB
cb7d27f29af6 [==============================================] 115b/115b
eb90ce17dcdf [==============================================] 185b/185b
8cd304396fa9 [==============================================] 200b/200b
0b9bd7b7539f [==============================================] 200b/200b
1f4bb64ae182 [========================================] 11.4MiB/11.4MiB
ef1f64844071 [========================================] 15.9MiB/15.9MiB
e95857abebe9 [========================================] 16.3MiB/16.3MiB
2020/03/06 11:40:45 terraform init command failed.
Error: exit status 1
Output:
Initializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Checking for available provider plugins...
Provider "artifactory" not available for installation.
A provider named "artifactory" could not be found in the Terraform Registry.
This may result from mistyping the provider name, or the given provider may
be a third-party provider that cannot be installed automatically.
In the latter case, the plugin must be installed manually by locating and
downloading a suitable distribution package and placing the plugin's executable
file in the following directory:
terraform.d/plugins/linux_amd64
Terraform detects necessary plugins by inspecting the configuration and state.
To view the provider versions requested by each module, run
"terraform providers".
Error: no provider exists with the given name
This can't work as it won't have my terraform_source with the provider binary.
I have resorted to building support for the provider into the resource image:
FROM golang:1.14 as builder
WORKDIR /workspace
ENV CGO_ENABLED="0"
RUN git clone https://github.com/atlassian/terraform-provider-artifactory.git \
&& cd terraform-provider-artifactory \
&& go build
# Terraform 0.12.22
FROM ljfranklin/terraform-resource@sha256:d63ba8bb3262392adfa3e68dad9b94584bcb4ec74095392f51a3a71543afa33a
COPY --from=builder /workspace/terraform-provider-artifactory/terraform-provider-artifactory /usr/local/bin/terraform-provider-artifactory
@Krenair I have a test here which exercises the custom plugin flow. I just updated that test to use the new backend syntax and it seems to be working as expected.
I remember Terraform being very picky about the directory and file names in the plugin directory. Was your filename exactly terraform.d/plugins/linux_amd64/terraform-provider-artifactory_v1.6.3
? To further debug the issue I would try replicating your setup locally and use terraform init --plugin-dir your-plugin-dir
to figure out whether Terraform is able to recognize your plugins correctly.
@Krenair oh, I missed the part about this being on the get
step. I didn't realize Terraform had this behavior, but it looks when you run terraform init
with backends it also tries to look up plugins to download from the default
workspace. So I'm guessing at some point you've ran an apply
against the default
workspace and Terraform is trying to be overly helpful by downloading the artifactory plugin that the default
workspace uses.
I'll do some thinking about how to fix this going forward. In the meantime you can work around the issue by baking the plugin into a custom docker image as you've done or by deleting or moving the statefile out of the default
workspace.
I think the fix for the resource will be to add -get-plugins=false
to the init
call for the get
and check
steps. This wouldn't work if you had a backend that was declared in a plugin, but it looks like support for plugable backends isn't happening soon so I'm fine with this fix for now.
Afraid not, the Concourse design doesn't allow get
steps to receive any directories as input: https://concourse-ci.org/implementing-resource-types.html#resource-in
@Krenair I think this change should fix the issue you were having. That commit is now present on the ljfranklin/terraform-resource:latest
and ljfranklin/terraform-resource:0.12.23
images, when you get a chance can you check if that fixes the error you were seeing?
Seems to have done the trick, thanks.