AWS ECS with CloudFormation
In a previous post, I went through the steps of setting up continuous integration and delivery using Gitlab and Amazon ECS. In this quick update, I will simplify the infrastructure provisioning using AWS CloudFormation.
This is my first time using CloudFormation directly. The learning curve was pretty short and the documentation is good. The only slight problem I encountered was when AWS was trying to create the ECS Service before the ELB listener had been created. It was a pretty quick fix to manually add a dependency.
The only other slight difference is that CloudFormation seems to give you less control over certain aspects of your infrastructure. For example, you can’t name the ECS cluster, task or service. The names are generated for you using and prepended with the CloudFormation stack name.
My previous post has a very enterprisy diagram that can help you visualize the archtitecture.
Here is my CloudFormation stack definition. It is a little verbose at 214 lines.
CloudFormation stack template
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Create an ECS cluster with an Application Load Balancer.",
"Parameters": {
"VpcId": {
"Type": "AWS::EC2::VPC::Id",
"Description": "Id of your VPC."
},
"DefaultSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup::Id",
"Description": "Id of the default security group for your VPC."
},
"Subnet1": {
"Type": "AWS::EC2::Subnet::Id",
"Description": "A subnet of your VPC."
},
"Subnet2": {
"Type": "AWS::EC2::Subnet::Id",
"Description": "A different subnet of your VPC"
},
"GitlabAuthToken": {
"Type": "String",
"Description": "The encoded authentication token for Gitlab. Ex: VGhpcyBpcyBub3QgYSByZWFsIHRva2VuLiBUaGlzIGlzYXQ=",
"NoEcho": true
},
"DockerImagePath": {
"Type": "String",
"Description": "The path to the inital docker image to deploy. Ex: registry.gitlab.com/[user]/[repo]"
}
},
"Resources": {
"ELBInboundSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Allow inbound HTTP from the world.",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
}
],
"VpcId": {
"Ref": "VpcId"
}
}
},
"LoadBalancer": {
"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties": {
"Name": "ecs-test-service-loadbalancer",
"Scheme": "internet-facing",
"SecurityGroups": [
{
"Ref": "DefaultSecurityGroup"
},
{
"Fn::GetAtt": [
"ELBInboundSecurityGroup",
"GroupId"
]
}
],
"Subnets": [
{
"Ref": "Subnet1"
},
{
"Ref": "Subnet2"
}
]
}
},
"TargetGroup": {
"Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
"Properties": {
"Port": "80",
"Name": "ecs-test-target-group",
"Protocol": "HTTP",
"VpcId": {
"Ref": "VpcId"
}
}
},
"Listener": {
"Type": "AWS::ElasticLoadBalancingV2::Listener",
"Properties": {
"DefaultActions": [
{
"Type": "forward",
"TargetGroupArn": {
"Ref": "TargetGroup"
}
}
],
"LoadBalancerArn": {
"Ref": "LoadBalancer"
},
"Port": 80,
"Protocol": "HTTP"
}
},
"EcsCluster": {
"Type": "AWS::ECS::Cluster"
},
"EcsInstance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-6bb2d67c",
"InstanceType": "t2.micro",
"Tags": [
{
"Key": "Name",
"Value": "test-ecs-thing"
}
],
"IamInstanceProfile": "ecsInstanceRole",
"SubnetId": "subnet-c5f371b2",
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash",
"\n",
"echo ECS_CLUSTER=",
{
"Ref": "EcsCluster"
},
" >> /etc/ecs/ecs.config",
"\n",
"echo ECS_ENGINE_AUTH_TYPE=dockercfg >> /etc/ecs/ecs.config",
"\n",
"echo ECS_ENGINE_AUTH_DATA={\\\"registry.gitlab.com\\\": { \\\"auth\\\": \\\"",
{
"Ref": "GitlabAuthToken"
},
"\\\" }} >> /etc/ecs/ecs.config",
"\n"
]
]
}
}
}
},
"EcsTask": {
"Type": "AWS::ECS::TaskDefinition",
"Properties": {
"ContainerDefinitions": [
{
"Cpu": 10,
"Essential": true,
"Image": {
"Ref": "DockerImagePath"
},
"Memory": 10,
"Name": "web-server",
"PortMappings": [
{
"ContainerPort": 80
}
],
"ReadonlyRootFilesystem": false
}
],
"Volumes": []
}
},
"EcsService": {
"Type": "AWS::ECS::Service",
"Properties": {
"Cluster": {
"Ref": "EcsCluster"
},
"DesiredCount": 1,
"LoadBalancers": [
{
"ContainerName": "web-server",
"ContainerPort": 80,
"TargetGroupArn": {
"Ref": "TargetGroup"
}
}
],
"Role": "ecsServiceRole",
"TaskDefinition": {
"Ref": "EcsTask"
}
},
"DependsOn": [
"Listener",
"EcsInstance"
]
}
},
"Outputs": {
"EcsService": {
"Value": {
"Ref": "EcsService"
}
},
"EcsCluster": {
"Value": {
"Ref": "EcsCluster"
}
},
"EcsTask": {
"Value": {
"Ref": "EcsTask"
}
}
}
}
Stack Temmplate Parameters
To create this stack, you first define some parameters. Here is an example file:
[
{
"ParameterKey": "VpcId",
"ParameterValue": "vpc-AAAAAAAA"
},
{
"ParameterKey": "DefaultSecurityGroup",
"ParameterValue": "sg-AAAAAAAA"
},
{
"ParameterKey": "Subnet1",
"ParameterValue": "subnet-AAAAAAAA"
},
{
"ParameterKey": "Subnet2",
"ParameterValue": "subnet-BBBBBBBB"
},
{
"ParameterKey": "GitlabAuthToken",
"ParameterValue": "VGhpcyBpcyBub3QgYSByZWFsIHRva2VuLiBUaGlzIGlzYXQ="
},
{
"ParameterKey": "DockerImagePath",
"ParameterValue": "registry.gitlab.com/YOUR-USER-NAME/YOUR-PROJECT-NAME"
}
]
Create stack on the command line
Creating and updating the stack via the command line:
# Create the stack
aws cloudformation create-stack --stack-name YOUR-STACK-NAME --template-body file://formation.json --parameters file://cfparameters.json
# Update the stack
aws cloudformation update-stack --stack-name YOUR-STACK-NAME --template-body file://formation.json --parameters file://cfparameters.json
# Describe the stack
aws cloudformation describe-stacks --stack-name YOUR-STACK-NAME
Running these commands will create AWS resources. Don’t forget to delete the stack when you are done.
Summary
CloudFormation was easy to learn and straightforward. The JSON is a bit verbose and I think you should actively manage the complexity of your stacks templates. They could quickly become unwieldy.
Next Steps
One thing that could be added to this stack is using an autoscaling group to spin up new ECS instances. Also, like the previous post, this stack is passing the docker authentication via EC2 user data – this should be moved to S3. I’m not totally clear on the best way to update to a new ECS task version. Should you update through CloudFormation, or through the ECS service directly? I’m not sure if there is benefit either way.