I recently had the opportunity to test out Terraform Cloud (TF Cloud), after many years of using Terraform. These are my first impressions.
*Continued in part 2
*Continued in part 3
I recently had some downtime with Greyshore Associates and wanted to test out Terraform Cloud. It has always been there on the periphery of my mind, but I just never had the chance to play around with it.
Getting Started
I proceeded to create a new empty Github repo and then copied over some some sample Terraform files from another repo. I generally just use these as a starting template and they are super basic, usually just creating some resource groups (RGs) in Azure.
Usually I use CircleCI, but in this case Terraform Cloud will be invoking the Terraform code which means we don't need the CircleCI config file.
After the repo was created and the Terraform code was reworked a bit to make it unique to this testing, I created an account for myself and an organization in Terraform Cloud. Creating an account and org are free. As well as utilizing base Terraform Cloud features. I will cover costs in more detail below.
Since other advanced features were out of scope for my testing, I stuck with the "Free" plan.
Workspaces
Next up, I created a new Workspace. One thing to note here is that a Terraform Cloud workspace is not the same as a general Terraform workspace. General Terraform workspaces are centered around TF state and use the same TF configuration files. Terraform Cloud workspaces are more like separating both your Terraform state and your TF configuration files by directory.
Here is a comparison chart on how TF Cloud workspaces compare to separate working directories in local Terraform:
When you create a new workspace, you need to choose a workspace type. There are three currently:
Version Control Workflow - When you store your TF config files in a git repository.
CLI-driven workflow - You use Terraform's CLI to execute runs in Terraform Cloud.
API-driven workflow - This is when you want a larger, parent pipeline to trigger TF Cloud using the Terraform API (generally, you would use a CI system to monitor code changes and then trigger TF Cloud once changes occur).
Since I was doing basic testing to get familiar with TF Cloud, I went with the Version Control Workflow workspace (this is the most common option for TF Cloud). Currently, you can connect to Github, Gitlab, Bitbucket, and Azure DevOps. There are sub-options for each, depending on the version you are using and your deployment methodology.
There are two different ways to connect to Github.com. Previously, you had to create an Oauth app and exchange secrets to get the connection setup. Now there is a largely "configuration free" method where you can authorize TF Cloud with Github.com. As you are going through the authorization process, you have to decide what repos you are granting TF Cloud access to. For me, I only granted it access to the new repo I made. After you do this, the workspace will be created in TF Cloud.
Workspace Configuration
Now that the workspace was setup, it pulled in the Terraform configuration from Github.com. But there was more work to be done before I could run a Terraform Plan. Specifically, the TF Cloud execution engines would not have access to Azure. Generally, I pass secrets in via (sensitive) environment variables, when not working with a full vault-like solution. For other testing, that is usually done with CircleCI and the env variables are passed into the Terraform specific steps of the pipeline. Since I was not using CircleCI in this case, then I had to make use of the TF Cloud variables.
If you look near the top of the workspace, you will see a horizontal menu. You can select "Variables" there and go to the variable page to configure them accordingly.
The variables on the page are split into two types:
Terraform Variables - these are variables that are defined in the Terraform configuration files.
Environment Variables - these are the same as env variables in a system shell.
I got to work creating the same environment variables that I was using in CircleCI to create allow the execution engines to establish a connection to the Azure tenant and subscription I was using.
One thing to note is that you can mark specific env variables as "sensitive" to obscure them from any console outputs, the variables page, and to prohibit any copy function from the variables page.
Terraform Plan and Apply
I went back to the overview page for the workspace and then clicked actions -> start a new plan. Unfortunately, the plan errored out.
The issue was simple, the defined variables in the TF configuration where not getting their values. Being unfamiliar with TF Cloud made the resolution a bit harder to get to, though.
At that time, I was defining the values for "subscription_id" and "resource_groups" in the terraform.tfvars file in Github. When running TF locally, this tfvars file gets pulled in automatically. After reading through the TF Cloud documentation, I found that TF Cloud generates a terraform.tfvars file based on the inputs on the variables page. I reasoned that my already defined file was probably getting ignored.
Now if I couldn't use the terraform.tfvars file for some reason, then I would normally go ahead and create another .tfvars file and then call it out in the Terraform plan and apply steps by using the --var-file flag. This isn't possible with how Terraform Cloud executes from source control. My initial reading gave me the impression that you had to define all variables in the web UI (on the variables page), which seemed strange to me.
I found this thread which shed a little more light on this and how you could avoid putting all variables in the UI manually. I'll address this in more detail in the next blog post, but for the sake of getting something going, I decided to input the two variables into the web UI.
For the subscription_id variable, this was just an easy string input. The other variable was using a map to create the RGs. When you create a new Terraform variable, there are several options. You can mark the value as sensitive (just like with the env variables) and there is also an option to use HCL. HCL allows you to throw TF code into the box and have it interpolate values during the run. I checked this box and through the whole map into the the value box.
After adding these values and invoking the plan again, it was successful.
After the plan completes successfully, you are then able to approve it. Once approved, the apply with run.
The plan output looks familiar to most console outputs, but the apply is a bit more unique within the UI (as shown below).
For both actions, you can download the raw job log.
Terraform State
Terraform Cloud will also manage your terraform state. In order to do this, all you need to do is omit a backend configuration within your TF config files.
Cost
Costs and feature comparison shown below, as of the date of this article:
Impressions
While I only have a few hours with Terraform Cloud under my belt, I am left with the following first impressions:
Positives
Not having to worry about storing and securing state on a backend is great.
Setup was very straight forward and easy for anyone familiar with Terraform.
Environment variable support is good and comparable to other CI/CD tools.
Free plan is great to get started and test out. The plan allows up to 5 users and never expires.
Concerns
I'm not seeing integration points with other common tools for Terraform code analysis, unless you incorporate them in a parent pipeline. Specifically, I'm talking about tools like Checkov, Terraform-Compliance, Terrascan, etc.
Terraform Cloud is for Terraform. Its not a full CI/CD tool, so if you need to incorporate other tasks, you will either need to have different pipelines and tools or use another CI tool to call Terraform Cloud. This can be additional overhead (operational costs, support, and administration).
Additional cost of using Terraform Cloud for advanced features and more than 5 users.
Terraform Cloud is easy to pick up and use. It could be useful for a team that is managing infrastructure only and is already using Terraform. With that said, there are trade offs and it may not make sense for some organizations that already have a full CI/CD tool in place. Also, it appears that Terraform Cloud is trying to drive you to use HashiCorp's other tools (like Sentinel) which incorporate natively into the TF Cloud workspaces. These HashiCorp tools cost additional money, obviously.
References
Terraform Cloud Documentation: https://www.terraform.io/docs/cloud/