Topic not found error raised when setting retention policies on multiple topics
esenac opened this issue · 11 comments
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Terraform Version
1.1.2
Affected Resource(s)
- pulsar_topics
Terraform Configuration Files
An example can be found here
Debug Output
https://gist.github.com/esenac/097a3bba59f36e169c6f8b378ed91da9
Panic Output
Expected Behavior
Topic retention policies (the default ones in this case) are correctly set on the topics after the topics have successfully been created.
Actual Behavior
Topic level retention policies are not set because the declared topics seem not to exist on Pulsar even if they're in the same Terraform file.
Steps to Reproduce
- Declare a list of topics
- Loop the list to create topics resources in Terraform
terraform apply
Important Factoids
The issue does not occur with a reduced amount of topics in the list.
References
@nodece not in the case we try to set policies to each topic, because the calls would be made and they would cause the "concurrency" problem.
@esenac I tried to run your project with the latest terraform-provider-pulsar, and it is working, the following is my log:
[+] Building 34.2s (16/16) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/apachepulsar/pulsar:2.7.3 1.8s
=> [internal] load metadata for docker.io/library/debian:stretch-slim 1.9s
=> [tf 1/3] FROM docker.io/library/debian:stretch-slim@sha256:5913f0038562c1964c62fc1a9fcfff3c7bb340e2f5dbf461610ab4f802368eee 0.0s
=> [stage-1 1/7] FROM docker.io/apachepulsar/pulsar:2.7.3@sha256:f858f9535668032db9d18c33b398b60a678696b944241c5213e4466f46c18fa7 0.0s
=> [internal] load build context 0.6s
=> => transferring context: 25.71MB 0.5s
=> CACHED [tf 2/3] RUN ARCH=$(if [ $(uname -m) = "aarch64" ]; then echo "arm64"; else echo "amd64"; fi) && apt-get update && apt-get i 0.0s
=> CACHED [tf 3/3] RUN chmod +x terraform 0.0s
=> CACHED [stage-1 2/7] COPY --from=tf terraform /bin/terraform 0.0s
=> [stage-1 3/7] COPY ./terraform-pulsar-plugin terraform.d/ 0.2s
=> [stage-1 4/7] COPY ./*.tf ./ 0.0s
=> [stage-1 5/7] COPY wait-for-it-api.sh ./ 0.0s
=> [stage-1 6/7] COPY ./conf/standalone.conf /pulsar/conf/standalone.conf 0.0s
=> [stage-1 7/7] RUN ./bin/pulsar standalone > /dev/null & export PROCESS_ID=$! && terraform init && ./wait-for-it-api.sh htt 31.0s
=> exporting to image 0.4s
=> => exporting layers 0.4s
=> => writing image sha256:2da0eb11693b1f5436375b7365042d59c535c607c5b2d3b0b1e3712f66c1f9fd 0.0s
=> => naming to docker.io/library/pulsar-examples 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
@nodece I've updated the provider and added retention policies to the topics in this commit, which led to the error. I've updated the examples repo accordingly on the topic-not-found-error branch.
The issue can be reproduced locally, but it can work on Pulsar 2.9, @esenac Should I add a loop to check whether the topic exists?
@codelipenghui Do you have any ideas to fix this?