Terraform & Azure ❤ GitLab CI — Part 3: Creating an Azure VM
With everything set up, we can start writing the code in Visual Studio Code, or your preferred editor. We’ll start by defining the network, disks, IP address and server we want to build.
This article is part of a series:
Variables
We’ll start out by defining a few variables. We’ll reuse things like the VM’s name several times, so it’s more convenient to store this in a variable. Variables can also be set or manipulated from your host, pipeline or Docker image by prefixing environment variables with TF_VAR_.
We’ll define these 4 variables to set up the VM. We can insert these variables to name the VM and its child resources. The defaults are present, so if we do not define these variables in GitLab or anywhere else, these values will be used. The only mandatory variable to manually set is DEFAULT_SSHKEY, or we won’t be able to login to our VM after creation.
Providers
To let Terraform know which platform we’re addressing and which modules to download, we’ll have to define a provider. In this demo, we’re using Microsoft Azure.
It’s a fairly straightforward block. We define the name and version of the provider, and any optional features we want. In the second block, we’ll describe where the Terraform state is stored. Substitute the above dummy configuration with data from the Storage Account you’ve created in the previous article to enable this backend.
The state keeps track of how we want our environment to look like and is used by Terraform to decide which actions to take during terraform plan and terraform apply.
The main course
We’ll now define the resources we want to create in main.tf. A Virtual Machine on Azure has several prerequisites that need to exist:
- Resource Group
- Virtual Network
- Subnet inside a Virtual Network
- Network Interface
- Public IP (optional)
You can copy the main.tf below into your code editor and adjust it to your liking, or attempt constructing your own from scratch. It will work as-is and create a cheap B-Series VM called “TestVM” in your Azure subscription.
As you can see, we use the var.VM_NAME in several places to keep the naming convention for each of the components in check. Since Terraform 0.12, you must reference variables in this manner:
name = var.VARIABLEWhen referencing variables inside of a string, you use ${}, as such:
name = "My name is ${var.VARIABLE}"To reference resource blocks, a different syntax is used. For example, you’ll create a Resource Group with location “West Europe.” Now, to create every new resource and manually type out “West Europe” would be redundant and cumbersome. You wouldn’t create all the child resources and closely related infrastructure in different regions, so being able to reference the Resource Group’s location makes life easier.
resource "azurerm_resource_group" "main" {
name = "${var.VM_NAME}-ResourceGroup"
location = var.LOCATION
}We then reference the location inside a different resource:
resource "azurerm_virtual_machine" "main" {
name = var.VM_NAME
location = azurerm_resource_group.main.location
}I’ve highlighted the relevant parts in bold. This type of variable references the type of resource (azurerm_resource_group), the unique name (main) of the resource and lastly, the value of the argument. (location)
Outputs
Most Terraform modules have one or more outputs related to the resource you’re creating. In this case, we need the public IP of the VM, so we can connect to it via SSH.
Building the pipeline
Now that we’ve prepared our Azure subscription and wrote the configuration for a VM and its child resources, we’ll still need to configure GitLab to deploy them automatically. The idea is that our pushes to a branch trigger a test run, and that a merge into master will start the actual deployment or alteration of resources.
In almost all CI/CD software (GitLab, Azure DevOps, BitBucket, etc.) a pipeline configuration is not much more than a script (much like Bash or PowerShell) with several if/else-esque blocks and preset functions. The syntax is usually YAML, and the bells & whistles differ per vendor.
Missing credentials
There’s still something missing. We’ve never entered our Microsoft account into GitLab, or created any sort of webhook that allows interaction between the two. Currently, our GitLab repository can’t communicate with our Azure subscription.
The Terraform Docker image will accept environment variables with access keys for your Storage Accounts and Azure subscriptions. Enter the appropriate values for the following variables under Settings — CI / CD — Variables:
ARM_ACCESS_KEY: The Access Key we got from the Storage Account.ARM_CLIENT_ID: The Client ID we got from the App Registration.ARM_CLIENT_SECRET: The Secret we’ve created in the App Registration.ARM_SUBSCRIPTION_ID: Your Subscription’s ID.ARM_TENANT_ID: The Tenant ID we got from the App Registration.TF_VAR_DEFAULT_SSHKEY: Your public SSH key.
Creating the configuration
To start rolling out resources automatically, we’ll need to create a configuration that defines what that looks like. It must be named exactly .gitlab-ci.yml and placed in the root of your project, or it will not be picked up by GitLab as a valid configuration. Below is a simple configuration that uses the official HashiCorp Terraform image:
This does a few things:
image:downloads the Terraform Docker image, to run all code from.before_script:runs a few preparatory lines of script and outputs the Terraform version to the logs.validate:runsterraform validate, which does a quick check on the syntax of your Terraform HCL. If it contains any errors, the pipeline will stop and show you the error.plan:runsterraform plan, which compares the current state against your current environment.artifact:will make the plan file available for download/debugging.apply:runsterraform apply, which prompts Terraform to reconcile the difference between the state and the current infrastructure. This can create, change, replace, or destroy resources. This step is only performed when the master branch is changed.
The GitLab docs have a few more CI-file examples if you’re interested in CI for different programming languages.
Deployment & Debugging
Once your resources and configurations are finished, you can now push your changes to a new branch on your repository. This should fire the validate and plan steps of your .gitlab-ci.yml. You’ll find any running pipelines in the GitLab web interface, under CI / CD — Jobs.
If all goes well, the plan and validate steps will pass. You can then set up a Merge Request, and merge your branch into the master branch, which will also trigger the apply step, applying your configuration to your Azure subscription.
Connecting to the VM
If everything has gone well, our apply step under Jobs will have an output at the end of the log, listing the public IP address of the server. To connect to our newly minted server, we use the value of admin_username from main.tf and set up an SSH session.
ssh azure-admin@ip.address.goes.here
Problems?
We may possibly run into errors during these steps. Perhaps there was a typo. Maybe we forgot to set up a container for the state storage. A missing variable, perhaps. The Jobs logs will tell us exactly what’s wrong, and provide clues. While still in the CI / CD — Jobs window, click on the step that has an error marking next to it. Carefully study the output and correct the issue.
Closing Statement
I hope this is enough of a basic guide to get you started and excited to work with IaC, Git and Terraform. The resource you’ve created is by no means production-worthy, since we’re missing essential things such as firewalling, redundancy, scaling, diagnostics, access control, monitoring and many other integral parts of running secure, functional infrastructure. When you’re done refining your infrastructure, take a look at Ansible to further manage your configurations.
If you’ve created something cool or have any questions or spotted an error/bug, please let me know in the comments :D
Thanks to Daniel Koopmans for valuable feedback and the many wacky Linux, Kubernetes and virtualization adventures.
