Amazon CloudFormation (an Introduction)

CloudFormation is Amazon Web Services’ core tool for expressing infrastructure as code. Infrastructure is described in either JSON or YAML (the keywords are nearly identical), and a template designer can help assemble resources and their dependencies. The specifics of each resource still have to be written out by hand.

From what I’ve seen, there is no limit on the kinds of resources you can define. Every resource type AWS offers can be coded, created and provisioned. You can set an explicit creation order (e.g. resource A before resource B) on top of the implicit order CloudFormation works out itself (e.g. an IAM role attached to an EC2 instance is always created first). Rollback is clean too: deleting a stack removes everything the stack created, and nothing is left behind.

That same design has a downside: working with resources that already exist is awkward. For example, if you define a policy for an existing S3 bucket, it completely overwrites any policy already on the bucket, and deleting the stack leaves the bucket with no policy at all. There are workarounds using Lambda functions, which are not covered here.

A stack file, in JSON or YAML, has three main sections:

• Parameters: the inputs, e.g. the SSH key pair to use with newly provisioned EC2 servers.
• Resources: the contents of the stack (EC2 servers, security groups, Route 53 DNS entries, RDS databases, etc.).
• Outputs: typically public IP addresses, but possibly other values depending on the resources created.

Note: According to the template anatomy documentation, Resources is the only section that is strictly required. A template with no inputs and no outputs seems very limiting, though.

Constants (AMI IDs, node names, etc.) can be placed in a Mappings section. Conditions let you make decisions, such as creating or skipping certain resources, based on things like the AWS Region or user-supplied Parameters.

As a coder at heart, I was drawn to the programmatic features CloudFormation offers. Here are a few JSON examples.

{
  "Parameters" : {
    "KeyName": {
      "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instances",
      "Type": "AWS::EC2::KeyPair::KeyName",
      "ConstraintDescription" : "must be the name of an existing EC2 KeyPair."
    },
    "InstanceType" : {
      "Description" : "EC2 instances type",
      "Type" : "String",
      "Default" : "t2.micro",
      "AllowedValues" : [ "t2.micro", "m4.large", "c4.large", "r3.2xlarge" ],
      "ConstraintDescription" : "must be a valid EC2 instance type."
    },
  }
  "Resources": {
    "MyEC2Instance" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
        "InstanceType" : { "Ref" : "InstanceType" },
        "KeyName" : { "Ref" : "KeyName" },
      }
    }
  }
}

{
  "Mappings": {
    "InstanceValues" : {
      "Names": { "MyInstanceName" : "myec2.environment.local" }
    }
  }
  "Resources": {
    "MyEC2Instance" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
        "Tags" : [ {
          "Key" : "Name",
          "Value" : { "Fn::FindInMap" : [ "InstanceValues", "Names" , "MyInstanceName" ] }
        } ],
      }
    }
  }
}

{
  "Parameters": {
    "S3BucketName": {
      "Description" : "Name of an existing S3 bucket",
      "Type": "String"
    },
  }
  "Resources" : {
    "EC2RolePolicy": {
      "Type": "AWS::IAM::Policy",
      "Properties": {
        "PolicyName": "MyEC2RolePolicy",
        "PolicyDocument": {
          "Statement": [ {
            "Effect": "Allow",
            "Action"   : [
              "s3:GetObject"
            ],
            "Resource": { "Fn::Join": ["", ["arn:aws:s3:::", { "Ref" : "S3BucketName" }, "/*"] ] }
          } ]
        },
      }
    }
  }
}

{
  "Resources": {
    "EC2Role": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "AssumeRolePolicyDocument": {
          "Statement": [{
            "Effect": "Allow",
            "Principal": { "Service": "ec2.amazonaws.com" },
            "Action": [ "sts:AssumeRole" ]
          }]
        },
        "Path": "/"
      }
    },
    "MyBucketPolicy" : {
      "DependsOn": "EC2Role",
      "Type": "AWS::S3::BucketPolicy",
      "Properties": {
        "Bucket": {"Ref" : "MyBucketName"},
        "PolicyDocument": {
          "Statement": [{
            "Action": ["s3:GetObject"],
            "Effect": "Allow",
            "Resource": { "Fn::Join": ["", ["arn:aws:s3:::", { "Ref" : "S3BucketName" } , "/*" ]]},
            "Principal": { "AWS": [ { "Fn::GetAtt" : [ "EC2Role" , "Arn" ] } ] }
          }]
        }
      }
    },
  }
}

More functions are available in CloudFormation templates; see the intrinsic function reference.

There are two main points of contact with tools such as Chef, Ansible, Puppet or Salt:

• Instance user data: the bash provisioning script passed to newly created instances. With the constructs above, you can inject node names, fixed IP addresses, S3 bucket names, public SSH keys and more.
• Security groups: you can open ports for configuration management engines that use protocols other than SSH (e.g. HTTPS for Chef).

The recommended practice is to keep CloudFormation focused on defining infrastructure, including installing and configuring base system packages, and then hand control over to your configuration management tool of choice. That tool should take care of installing and configuring the services themselves.

That’s all for now. Thanks for reading!

Note: This article was written by an AWS Certified Solutions Architect (Associate). Always work with an expert when setting up production environments.