Add support to Bitbucket
Closed this issue · 13 comments
Hi,
It would be nice to support Bitbucket as a source code repository for Jenkins Docker image.
We can take 2 different approaches;
- Fork current module that supports Github and modify it to work with Bitbucket
- Add support to Bitbucket in the module and have a variable to control which one to use
The module in question is https://github.com/cloudposse/terraform-aws-cicd
I would like to hear your inputs @osterman and @aknysh because you have a more broader view about the solution and its directions.
Thanks in advance,
Ivan Pinatti
@ivan-pinatti thanks for the request.
Let me review CodePipeline and how it could support BitBucket and CodeCommit.
I'll get back to you shortly.
Hi @aknysh,
A possible solution is to have a variable to select between Github/Bitbucket in this module and change the following variables to generic names;
- github_oauth_token to git_token
- github_organization to git_organization
- github_repo_name to git_repo_name
- github_branch to git_branch
This add-on would required logical changes in the following modules;
- https://github.com/cloudposse/terraform-aws-cicd
- https://github.com/cloudposse/terraform-aws-codebuild
We could use Terraform conditionals to control the flow and implement the choosen one.
In this manner it wouldn't require to fork these modules, however, it would break current implementations because of the github_oauth_token variable renaming. The other mentioned variables already have generic names in the subsequent modules.
Basically, it will introduce changes in the variables passing from one module to another, modifications in the CodeBuild and CodePipeline Terraform resources declaration / logic.
Lines affected;
Jenkins
- https://github.com/cloudposse/terraform-aws-jenkins/blob/master/main.tf#L123-L126
- https://github.com/cloudposse/terraform-aws-jenkins/blob/master/examples/new_vpc_new_subnets/main.tf#L29-L32
- https://github.com/cloudposse/terraform-aws-jenkins/blob/master/examples/existing_vpc_existing_subnets/main.tf#L29-L32
- https://github.com/cloudposse/terraform-aws-jenkins/blob/master/examples/existing_vpc_new_subnets/main.tf#L29-L32
CI/CD
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/main.tf#L150
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/main.tf#L192
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/main.tf#L197
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/main.tf#L262
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/main.tf#L267
CodeBuild
Let me hear your thoughts on this after you analysis.
Cheers,
Ivan Pinatti
I forgot to mention these lines too;
Jenkins
CI/CD
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/variables.tf#L33-L35
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/variables.tf#L38
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/variables.tf#L42
- https://github.com/cloudposse/terraform-aws-cicd/blob/master/variables.tf#L46
CodeBuild
Seems like a reasonable approach to me (renaming vars and using ternaries). Are there any additional variables that need to be added in order to support BitBucket?
I am alright with not maintaining backward compatibility since all modules have releases to ensure stability.
@osterman, by what is described in Terraform CodeBuild and CodePipeline resource documentation it doesn't look like it will require any. Although, it is not expressly declared that supports Bitbucket, specially in the CodePipeline page.
Let's wait for @aknysh inputs as well.
It looks like CodePipeline does not support BitBucket as Source (at least I could not find any mention of it in any docs). Maybe it could 'just work' as GitHub? Needs to be tested.
It does (officially) support S3, CodeCommit and GitHub:
https://docs.aws.amazon.com/codepipeline/latest/userguide/integrations-action-type.html
https://docs.aws.amazon.com/codepipeline/latest/userguide/reference-pipeline-structure.html
CodeBuild, on the other hand, does support BitBucket as Source (they added it not too long ago)
https://aws.amazon.com/about-aws/whats-new/2017/08/aws-codebuild-now-supports-atlassian-bitbucket-cloud-as-a-source-type/
https://stackoverflow.com/questions/41686602/im-trying-to-integrate-bitbucket-into-aws-code-pipeline-what-is-the-best-appro
If we want to use CodePipeline as the Source for CodeBuild (as we do now in our CI/CD module), then BitBucket is not supported in this mode. There are two ways to solve this if needed:
-
Mirror a BitBucket repo to a CodeCommit repo, and then use CodeCommit as Source in CodePipeline (should be an easy change): https://blog.frugalops.com/ci-and-cd-with-aws-codepipeline-codebuild/
-
Use Webhooks to notify a remote service by issuing an HTTP POST when a commit is pushed to the repository. AWS Lambda receives the HTTP POST through Amazon API Gateway, and then downloads a copy of the repository. It places a zipped copy of the repository into a versioned S3 bucket. CodePipeline can then use the zip file in S3 as a source. https://aws.amazon.com/blogs/devops/integrating-git-with-aws-codepipeline/
These approaches with CodePipeline do not look ideal (too many moving parts).
We'll have to look how to integrate BitBucket with CodeBuild. We will have to eliminate the first stage in the CodePipeline (Source) and just leave the other two stages (Build and Deploy), and configure CodeBuild to use a Source from S3, BitBucket, GitHub or CodeCommit. All of these need to be tested.
@ivan-pinatti If you get stuck trying to integrate BitBucket with CodeBuild, let us know. I suggest manually testing all the components on AWS before trying to automate it just to know that it works with BitBucket
Thanks
CodePipeline doesn't support Bitbucket. When I change the source in CodeBuild options in AWS console it shows right away a warning message on the top, see it below;
I've researched AWS forums and there are some threads asking for the feature to support Bitbucket, I made a +1 comment to give AWS more visibility and to keep me posted about updates. I suggest you guys do the same. The thread is this one
In my opinion we can leave this issue open until AWS releases it, I don't think that it will take much longer.
@ivan-pinatti
thanks
We can wait for AWS to add BitBucket to CodePipeline.
At the same time you could test CodeBuild with BitBucket, it does support it.
Yes, it is working great with CodeBuild and manual deploy to Elastic Beanstalk for now.
Let's wait for AWS to release the feature.
Thanks.
Any updates on this feature? Seems it is taking a while :)
will close for now.
re-open if need BitBucket support and CodePipeline supports it
Bitbucket support for CodePipeline is now in beta and is available to test.
https://aws.amazon.com/about-aws/whats-new/2019/12/aws-codepipeline-now-supports-atlassian-bitbucket-cloud/
@aknysh can we reopen this issue ?