“If you’re doing something more than twice a month — automate it!” — said to me my DevOps boss every time I used to do some repetitive stuff by hand. My first automation was the bash script deploying a WordPress infrastructure in AWS. It worked perfectly, but the experience was similar to cutting the huge tree with the pocket knife.
As I gained more experience, I realized there are more problems with the manual configuration approach. If you ever had a chance to set up and wire together even a small number of AWS service, like several EC2 instances spread across a few public and private subnets, a database, an S3 bucket, etc., you know how many small but significant details you should keep in mind to make this system work. But the real problems come when you need to replicate the same configuration from a test environment to production, and then you need to keep those environments’ configurations in sync, track configuration change, etc.
There certainly must be some tools to handle the highlighted issues. So, after spending some time in research, to understand the basics, the adventure has begun š
IaC to the Rescue
The problems above are easily handled with the approach called “Infrastructure as a code (IaC)”. The idea is to declaratively describe your infrastructure, store the configuration in a source control system like Git, and automatically update a running system whenever the configuration changes.
Nowadays, there are plenty of tools in the IaC domain operating on a different level and supporting different environments both on-premise and in a cloud. In this article, we will help you understand the basics of the AWS CloudFormation which is a managed IaC service provided by Amazon specifically for their AWS cloud platform.
Prerequisites
We expect you have an AWS account, are familiar with the most famous AWS service and concepts such as IAM, EC2, AMI, SSH Keys, S3, VPC, subnets, etc., and have at least some experience configuring these services manually via AWS Console or using AWS CLI.
Write a CloufFormation Template
So, let’s see how can we create an EC2 instance in a default VPC and subnet. First of all, you have to create a configuration file known as a template. CloudFormation template format is either json or yaml. In this article, we will use yaml as a less verbose option.
So, let's create a test-cf.yml file in your favorite text editor or an IDE of your choice and put the following code there:
Yes, it is that simple! Now, we need to instruct AWS to create an environment, also known as Stack, using our template. A stack is a collection of AWS resources that you can manage as a single unit: create, update, or delete. All the resources in a stack are defined by the CloudFormation template.
Turn the Template into a Stack
To create a stack with new resources from the template, navigate to the CloudFormation service in the AWS Console, and click the Create Stack button. Select the Template is ready and Upload a template file options.
You might have noticed that it is possible to choose a template stored in an S3 bucket. In fact, even if you choose to upload a template file, it will be stored in an S3 bucket automatically and the CloudFormation will refer that copy of the template.
On the next step, enter the stack name and follow the wizard, leaving the rest of the options with their default values. Finally, press Create stack on the bottom of the confirmation page. If you did everything correctly, after a few minutes you will have an EC2 instance up and running.
Fine-grained Configuration
The problem with our instance is that there is no way to log into it. Let's try to enable SSH access. For this, you need an SSH key attached to the instance. You cannot automate SSH key creation within the template for multiple reasons, so you have to create one manually in the EC2 console. Don't forget to download the key to your workstation. Then add the following line to the Properties object of your instance:
KeyName: <the name of your key>
Be careful with formatting, as the yaml format is highly sensitive to the formatting issues, especially to the number of spaces in every line.
Let’s also consider other properties before updating our stack. If you need to customize your storage settings, consider the following snippet:
You may also want to specify the subnet, security group, or a role your instance will have:
Now we need to re-upload the template. Go to the CloudFormation console, select your stack and click Update, select Replace current template and Upload a template file and go through the wizard. On the bottom of the Summary page (the last step), you will see the list of resources to be modified. Confirm the changes and wait until the changes are applied.
Now you have a customized EC2 instance that you can enter into via SSH using the specified key.
A good thing about AWS is that they provide you with excellent documentation for CloudFormation Template Anatomy in general and the supported EC2 configuration properties particularly.
Fight with Hardcode
Now, the problem is that several values, including SSH key name, VPC ID, Subnet, Security Group, and the IAM role are all hardcoded.
What if you don’t know the values upfront, or if you want to create the resources and refer them by IDs dynamically?
First of all, you may specify Parameters in the template. For every parameter, there will be a user input generated. Depending on the parameter type, the input could be just a plain text field, a drop-down with a predefined list of values, or even an automatically filled drop-down. E.g., if you want to replace hardcoded SSH key name with a text field, add the following code to your template:
And if you want CloudFormation to generate a drop-down list with all available keys, specify the type as follows:
Type: AWS::EC2::KeyPair::KeyName
Now you have to replace the hardcoded value with the reference to the parameter. To do this, just specify the KeyName property in the Ec2Instance object like this:
KeyName: !Ref KeyName
In addition to parameters, you may also refer to other built-in properties or even other objects. E.g. you may tag your instance with your stack name, or even combine your stack name with other values using the !Join Intrinsic Function:
You may also want to add a drop-down list with available availability zones, and, of course, you don’t want to hard-code them. Again, you can achieve that using multiple Intrinsic Functions, such as Select, Ref and GetAZs:
Summary
In this article, we spotlight just a tip of the tip of the CloudFormation iceberg, but this should give you an understanding of how easy and convenient it is to adopt the IaC approach and turn your infrastructure configuration into an easily maintainable, versioned form.
In our future articles, we will focus on particular features of the CloudFormation. Stay tuned!
P.S. Here is the full content of the template:
