Amazon CloudFormation (an introduction)

CloudFormation is the core functionality provided by Amazon Web Services in the area of expressing infrastructure as code. One can write the infrastructure design in either JSON and YAML (with similar syntax keywords); there is also a template designer that may help in putting together the infrastructure elements and their dependencies. The particular details of each resource being defined must be coded out, though.

There are no limitations on the types of resources that can be brought up, as far as I noticed – each resource type provided by Amazon can be coded and subsequently created and provisioned. One can also define an explicit order (e.g. some resource to be created before another), apart from the implicit order that can be deduced (e.g. if an EC2 instance is defined with a IAM Role, the role is always created before the instance). The revert process is also clean: deleting the CloudFormation stack does delete everything created by the stack itself; nothing gets left behind.

This design imposes another type of limitation, though – the interaction with pre-existing resources is not clean. Example: if one needs to define a particular policy for an existing S3 bucket, this policy will fully overwrite any existing bucket policy; the stack deletion will also leave the bucket with no policy. There are workarounds using Lambda functions, not covered in this particular text.

Template Structure

There are 3 mandatory sections in the stack file, be it JSON or YAML:

  • Parameters, describing the inputs, e.g. the SSH key pair that is to be used with freshly provisioned EC2 servers.

  • Resources: the stack contents (e.g. EC2 servers, security groups, Route 53 DNS entries, RDS databases, …)

  • Outputs – usually public IP addresses, but there might be more, depending on the resources created.

Note: per the template anatomy definition, the Resources section is the only one that is explicitly required. Not providing any input and not being explicitly interested in any output seems very limiting, though.

Constants (e.g. ami ids, node names) can be defined in a section called Mappings. One can also define Conditions in order to take decisions (e.g. create or not create certain resources) based on data such as the Amazon Region or the user provided Parameters.

Programatic Constructs

Me, being a coder at the very core, I got interested in the programatic constructs provided by CloudFormation. Let me put down a couple of examples (JSON):

  • Using user inputs (Ref):

      "Parameters" : {
        "KeyName": {
          "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instances",
          "Type": "AWS::EC2::KeyPair::KeyName",
          "ConstraintDescription" : "must be the name of an existing EC2 KeyPair."
        },
        "InstanceType" : {
          "Description" : "EC2 instances type",
          "Type" : "String",
          "Default" : "t2.micro",
          "AllowedValues" : [ "t2.micro", "m4.large", "c4.large", "r3.2xlarge" ],
          "ConstraintDescription" : "must be a valid EC2 instance type."
        },
    ...
    
      "Resources": {
    ...
        "MyEC2Instance" : {
          "Type" : "AWS::EC2::Instance",
          "Properties" : {
            "InstanceType" : { "Ref" : "InstanceType" },
            "KeyName" : { "Ref" : "KeyName" },  
    ...
    
  • Constants lookup (Fn::FindInMap):

      "Mappings": {
        "InstanceValues" : {
    	  "Names": { "MyInstanceName" : "myec2.environment.local" }
    ...
      "Resources": {
        "MyEC2Instance" : {
          "Type" : "AWS::EC2::Instance",
          "Properties" : {
    ...
    		"Tags" : [ {
             "Key" : "Name",
             "Value" : { "Fn::FindInMap" : [ "InstanceValues", "Names" , "MyInstanceName" ] }
    		} ],
    ...
    
  • Text aggregation (Fn::Join):

      "Parameters": {
      	"S3BucketName": {
    	  "Description" : "Name of an existing S3 bucket",
    	  "Type": "String"
    	},
    ...
      "Resources" : {
        "EC2RolePolicy": {
          "Type": "AWS::IAM::Policy",
          "Properties": {
            "PolicyName": "MyEC2RolePolicy",
            "PolicyDocument": {
              "Statement": [ {
                "Effect": "Allow",
                "Action"   : [
                  "s3:GetObject"
                ],
                "Resource": { "Fn::Join": ["", ["arn:aws:s3:::", { "Ref" : "S3BucketName" }, "/*"] ] }
              } ]
            },
    ...
    
  • Getting the ARN of another resource (Fn::GetAtt):

      "Resources": {
    ...
        "EC2Role": {
          "Type": "AWS::IAM::Role",
          "Properties": {
            "AssumeRolePolicyDocument": {
              "Statement": [{
                "Effect": "Allow",
                "Principal": { "Service": "ec2.amazonaws.com" },
                "Action": [ "sts:AssumeRole" ]
              }]
            },
            "Path": "/"
          }
        },
    ...
    	"MyBucketPolicy" : {
    	  "DependsOn": "EC2Role",
    	  "Type": "AWS::S3::BucketPolicy",
    	  "Properties": {
    		"Bucket": {"Ref" : "MyBucketName"},
    		"PolicyDocument": {
    		  "Statement": [{
    			"Action": ["s3:GetObject"],
    			"Effect": "Allow",
    			"Resource": { "Fn::Join": ["", ["arn:aws:s3:::", { "Ref" : "S3BucketName" } , "/*" ]]},
    			"Principal": { "AWS": [ { "Fn::GetAtt" : [ "EC2Role" , "Arn" ] } ] }
    		  }]
    		}
    	  }
    	},
    ...
    

There are more functions that can be used in a CloudFormation template (reference here).

Integrating with Configuration Management solutions

There are 2 main “points of contact” with solutions such as Chef, Ansible, Puppet or Salt:

  • Instance user data (the bash provisioning script that can be passed to newly created instances). Using the programatic constructs described before, one can inject node names, fixed ip addresses, S3 bucket names, public SSH keys (and so on);

  • Security groups: ports can be opened for configuration management engines that use connection protocols different from SSH (e.g. https for Chef).

The “good practice” is to limit CloudFormation to infrastructure definition, this including the installation and configuration of base system packages – and then to pass the control to the configuration management tool of one’s choice. That particular tool should go on with the service installation and configuration.

This ends this text. Thank you for your read!


Note: This text was written by an AWS Certified Solutions Architect (Associate). Please do always work with an expert when setting up production environments.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.