First steps with Terraform

The objective

The objective is to become knowledgable about cloud computing by trying many of the services in the AWS landscape. However, I want to do this in a manner that is repeatable and in a way that will allow me to build knowledge on top of what I have learnt in the past. It seems the obvious answer is to start with Infrastructure as Code. If everything I do from now on is coded, commented and version controlled, I will have a reference to look back on and build upon. However, I can’t capture everything in code, for example the decisions I make and the problems I face. Therefore I will use the medium of Medium to document and share my journey.

In the beginning

I drop a note to a friend who works in Cloud Infrastructure, asking him which toolset I should learn. He tells me to look at Terraform and Ansible. He says he’s asked Amazon about their own IaC product, CloudFormation, and “even they consider Terrafrom a tier 1 product and a cross-cloud provider”. So Terraform it is. I begin with opening an AWS account and then go to Hashicorp’s Terraform website. It’s Open Source, so that’s good. I note that there’s a Cloud version (free for up to 5 users!) and a CLI (Command Line Interface) version. I flick around the hero menu at the top of the page and note that there are Tutorials, Docs and a Community. It looks like you can download a binary to run from the command line on your desktop PC (I have a Mac) and view the source code on Github. I start by creating an account. Immediately it seems to be taking me down the Terraform Cloud route. I’m a stickler for punishment though and I think I want to use the CLI to learn this properly. I go back to the Tutorials and have a look at Get Started — AWS.

Following the tutorial

The tutorial is simple enough to follow. Some hurdles I encounter along the way:

• When I turn on command line auto-completion I find that my .bash_profile is owned by root. A quick chown via sudo resolves that.
• The first example encourages you to run an Nginx web server in a Docker container. The Docker download is half a gig, so I cancel that and just read the tutorial.
• Installing the AWS CLI was easy. However, I realised now was a good time to create an IAM User in a group called admin with full administrator access — rather than using my AWS root account. I also add MFA using the Google Authenticator app. This took a while to set up.
• I wonder if everyone forgets to write down their secret key when they create their access key.
• I set the output of aws configure to json as I don’t know any better.
• After using the map constraint I start getting a warning: “Quoted type constraints are deprecated”. I’ll need to look at the documentation to fix this.
• It turns out I can use the Terraform Cloud account — I can store my state data there, which is pretty nifty.

At this point it all starts to go wrong, but it is late, so I decide to go to bed and start again the following day. However, I do remember to destroy my EC2 instances before I leave.

Final steps

The following day I delete my workspace from the Terraform Cloud and start again. However, it seems the tutorial is not up-to-date. The state data can be transferred to the cloud, but you need to delete the state data from your machine before running terraform apply.

I then face another hurdle — my AWS credentials appear to have disappeared: “No valid credential sources found for AWS Provider”. I check for them by running aws configure. I look in the file where they’re supposed to reside (..\.aws\credentials). All is well, so why is it erroring? A quick Google leads me to this bug on Github, where alvarodelvalle tells me I need to set the credentials as variables in the workspace in the Terraform Cloud. We’re back in business. Thank you Alvaro!

What I’ve learnt

Here’s a quick diagram to show how it all fits together in my head.

The key commands to run are:

terraform init This initialises the configuration and downloads the provider plugin — i.e. in this case the code to communicate with the AWS CLI.

terraform apply This applies the configuration and builds your infrastructure

You can also use terraform fmt to reformat your *.tf files so they look neat and tidy, and terraform validate to check the config is valid.

Once your infrastructure has been created you can use terraform show to see the current state.

Finally, you can shut it all down (and stop incurring costs if you’re not on a free AWS tier) by running terraform destroy.

In conclusion

I’ve created an AWS account and an IAM user to manage it. I’ve installed the AWS CLI on my Mac and configured it to log in to my AWS account as my IAM user. I’ve then installed Terraform and configured it to provision and run a t2.micro EC2 instance by calling the AWS CLI. Finally, I created a Terraform Cloud account and configured the Terraform CLI to save any state data in a workspace there.

Next steps: fix the map constraint error; find out how I connect to the EC2 instance; see if I can install something on the EC2 instance using Terraform.