/terraform-jenkins-aws-fargate

Terraform stack to deploy Jenkins on ECS Fargate with Jenkins configuration stored in EFS and agents on ECS fargate

Primary LanguageHCL

Terraform Jenkins AWS ECS Fargate

Terraform stack to deploy Jenkins on ECS Fargate with Jenkins configuration stored in EFS and agents on Fargate. It can be used as a starting point to build a production ready Jenkins on AWS.

More details can be found on my blog post (in French).

How it works

It uses a docker image based on the official Jenkins. See docker/ folder.

The following main resources will be created:

  • An application load balancer in front of Jenkins.
  • A network load balancer for Agent -> Controller communication. For more information about how Controller <-> Agents communication works, see this page.
  • An EFS to store Jenkins configuration.
  • An S3 bucket used by the Jenkins Configuration as code plugin to get the configuration generated by Terraform.
  • Two log groups for the Controller and agents logs

Architecture

Prerequisites

  • A VPC with public and private subnets configured properly (route table, nat gateways...)
  • An IAM user with the proper policies to run Terraform on the following services: EC2, ECS, IAM, S3, Cloudwatch, EFS, Route53 et ACM.
  • A recent version of Terraform ( > 0.12.20)

The only required Terraform variables are:

  • vpc_id : the VPC ID
  • public_subnets : public subnets IDs
  • private_subnets : private subnets IDs

See variables.tf for all the possible variables to override.

AWS authentication:

export AWS_PROFILE=...
# or
export AWS_SECRET_ACCESS_KEY=""
export AWS_ACCESS_KEY_ID=""

Deployment:

export TF_VAR_vpc_id="vpc-123456789"
export TF_VAR_private_subnets='["private-subnet-a", "private-subnet-b", "private-subnet-c"]'
export TF_VAR_public_subnets='["public-subnet-a", "public--subnet-b", "public-subnet-c"]'
terraform init
terraform apply

Get the default admin credentials and connect to the controller url in the output jenkins_public_url:

terraform output jenkins_credentials

The first time you access the controller, the Getting started guide will ask you to install the recommended plugins. Install them and restart the controller.

Docs

Requirements

Name Version
terraform >= 1
aws ~> 4
random >= 3

Providers

Name Version
aws ~> 4
random >= 3

Modules

No modules.

Resources

Name Type
aws_acm_certificate.controller_certificate resource
aws_acm_certificate_validation.validation resource
aws_alb.alb_jenkins_controller resource
aws_alb_target_group.jenkins_controller_tg resource
aws_cloudwatch_log_group.agents resource
aws_cloudwatch_log_group.jenkins_controller resource
aws_cloudwatch_metric_alarm.alb_healthy_host_count resource
aws_cloudwatch_metric_alarm.alb_too_many_5xx_errors resource
aws_cloudwatch_metric_alarm.efs_burst_credit_balance resource
aws_cloudwatch_metric_alarm.jenkins_high_cpu resource
aws_cloudwatch_metric_alarm.jenkins_high_memory resource
aws_ecs_cluster.cluster resource
aws_ecs_cluster_capacity_providers.capacity_providers resource
aws_ecs_service.jenkins_controller resource
aws_ecs_task_definition.jenkins_controller resource
aws_efs_file_system.jenkins_conf resource
aws_efs_mount_target.mount_targets resource
aws_iam_policy.controller_ecs_task resource
aws_iam_role.agents_ecs_execution_role resource
aws_iam_role.agents_ecs_task_role resource
aws_iam_role.controller_ecs_execution_role resource
aws_iam_role.controller_ecs_task_role resource
aws_iam_role_policy_attachment.agents_execution_policy resource
aws_iam_role_policy_attachment.controller_ecs_task resource
aws_iam_role_policy_attachment.controller_execution_policy resource
aws_lb.nlb_agents resource
aws_lb_listener.agents_http_listener resource
aws_lb_listener.agents_jnlp_listener resource
aws_lb_listener.controller_http resource
aws_lb_listener.controller_http_redirect resource
aws_lb_listener.controller_https resource
aws_lb_target_group.nlb_agents_to_controller_http resource
aws_lb_target_group.nlb_agents_to_controller_jnlp resource
aws_route53_record.alb_dns_record resource
aws_route53_record.certificate_validation_record resource
aws_s3_bucket.jenkins_conf_bucket resource
aws_s3_bucket_acl.example resource
aws_s3_bucket_versioning.conf_bucket resource
aws_s3_object.jenkins_conf resource
aws_security_group.alb_security_group resource
aws_security_group.efs resource
aws_security_group.jenkins_agents resource
aws_security_group.jenkins_controller_ecs_service resource
aws_security_group_rule.alb_egress_all resource
aws_security_group_rule.alb_ingress_http resource
aws_security_group_rule.alb_ingress_https resource
aws_security_group_rule.allow_agents_to_jks_jnlp_port resource
aws_security_group_rule.allow_jenkins_to_efs resource
aws_security_group_rule.controller_egress_all resource
aws_security_group_rule.from_private_nlb_network_interfaces resource
aws_security_group_rule.jenkins_agent_egress resource
aws_security_group_rule.jenkins_controller_ingress_alb resource
aws_sns_topic.alarms_topic resource
random_password.admin_password resource
aws_caller_identity.caller data source
aws_iam_policy_document.controller_ecs_task data source
aws_iam_policy_document.ecs_assume_role_policy data source
aws_network_interface.each_network_interface data source
aws_route53_zone.dns_zone data source

Inputs

Name Description Type Default Required
private_subnets Private subnets to deploy Jenkins and the internal NLB set(string) n/a yes
public_subnets Public subnets to deploy the load balancer set(string) n/a yes
vpc_id The VPC id string n/a yes
agent_docker_image Docker image to use for the example agent. See: https://hub.docker.com/r/jenkins/inbound-agent/ string "elmhaidara/jenkins-alpine-agent-aws:latest-alpine" no
agents_cpu_memory CPU and memory for the agent example. Note that all combinations are not supported with Fargate.
object({
memory = number
cpu = number
})
{
"cpu": 1024,
"memory": 2048
}
no
agents_log_retention_days Retention days for Agents log group number 5 no
allowed_ip_addresses List of allowed IP addresses to access the controller from the ALB set(string)
[
"0.0.0.0/0"
]
no
aws_region The AWS region in which deploy the resources string "eu-west-1" no
controller_cpu_memory CPU and memory for Jenkins controller. Note that all combinations are not supported with Fargate.
object({
memory = number
cpu = number
})
{
"cpu": 1024,
"memory": 2048
}
no
controller_deployment_percentages The Min and Max percentages of Controller instance to keep when updating the service. See https://docs.aws.amazon.com/AmazonECS/latest/developerguide/update-service.html
object({
min = number
max = number
})
{
"max": 100,
"min": 0
}
no
controller_docker_image Jenkins Controller docker image to use string "elmhaidara/jenkins-aws-fargate:2.338" no
controller_docker_user_uid_gid Jenkins User/Group ID inside the container. One should consider using access point. number 0 no
controller_java_opts JENKINS_OPTS to pass to the controller string "" no
controller_jnlp_port JNLP port used by Jenkins agent to communicate with the controller number 50000 no
controller_listening_port Jenkins container listening port number 8080 no
controller_log_retention_days Retention days for Controller log group number 14 no
controller_num_executors Set this to a number > 0 to be able to build on controller (NOT RECOMMENDED) number 0 no
default_tags Default tags to apply to the resources map(string)
{
"Application": "Jenkins",
"Environment": "test",
"Terraform": "True"
}
no
efs_burst_credit_balance_threshold Threshold below which the metric BurstCreditBalance associated alarm will be triggered. Expressed in bytes number 1154487209164 no
efs_performance_mode EFS performance mode. Valid values: generalPurpose or maxIO string "generalPurpose" no
efs_provisioned_throughput_in_mibps The throughput, measured in MiB/s, that you want to provision for the file system. Only applicable with throughput_mode set to provisioned. number null no
efs_throughput_mode Throughput mode for the file system. Valid values: bursting, provisioned. When using provisioned, also set provisioned_throughput_in_mibps string "bursting" no
fargate_platform_version Fargate platform version to use. Must be >= 1.4.0 to be able to use Fargate string "1.4.0" no
route53_subdomain The subdomain to use for Jenkins Controller. Used when var.route53_zone_name is not empty string "jenkins" no
route53_zone_name A Route53 zone name to use to create a DNS record for the Jenkins Controller. Required for HTTPs. string "" no

Outputs

Name Description
agents_log_group Jenkins agents log group
controller_config_on_s3 Jenkins controller configuration file on S3
controller_log_group Jenkins controller log group
jenkins_credentials Credentials to access Jenkins via the public URL
jenkins_public_url Public URL to access Jenkins

References: