Write automated tests for your Terraform code

When it comes to Azure, I always prefer to implement infrastructure-as-a-code through ARM template than Terraform, but that is not always a choice; It is called life. Love it or not, sometimes you need to implement your infrastructure-as-a-code with Terraform and believe me or not, maintaining a large codebase of Terraform is not the most interesting job in the world. Introducing a change in a infrastructure-as-a-code’s code is always error prone and hard to test. Well, it shouldn’t be. Terratest helps you to write automated tests for your Terraform code. A Go Library which helps you to test your Terraform code, Packer templates and Docker images.

The good news is, starting with Terratest is simple and straightforward. You just need to Install Go first (This was the hardest part for me to convince myself to install Go 🙂 ), and then you just need to add Terratest to your project through one of the Go dependency manager such as dep.

 dep ensure -add github.com/gruntwork-io/terratest 

You are all setup now. You just need to write your tests using Go’s built-in package testing and then use Terratest to execute your Terraform code to deploy the  infrastructure to your environment; And then, use Terratest tools to validate your infrastructure (E.g. Making HTTP or API calls). When you finished with your testings you simply need to tear down the environment.

This way you simply can apply changes to your Terraform code with confidence.

Stop doing “DO NOT MERGE” pull requests

You probably have been in a situation where you finished your task and ready to create a pull request, but for whatever reason you don’t want your changes to get through (For example, you want to merge your changes back into the main branch after the current release). A common behavior – or let’s say old fashioned way – is to put something like “DO NOT MERGE” or “WORK IN PROGRESS” in the pull request’s title. And as an unwritten rule you should write it in all caps :). In such a situation, you can draft a pull request instead of creating a pull request.

To Draft a Pull Request in Azure Repos, you need to choose “Create as draft” when you want to create a pull request.

If you look at the list of pull requests now, you will see your pull request marked with “Draft” token.

Bypass branch policy in Azure Repos

It’s a good practice to lock down your repo and put some branch policies in place to avoid merging unwanted codes and non-complaint branches and commits into your master branch. However, we’ve all been in that situation where something urgently needs a fix and for some reasons we can’t meet the branch policies. By using Azure Repos, you can give a special permission to a set of users to override and bypass branch policies when things are on fire.

Azure Repos allows your lead developers to bypass branch policies when they want to push code or complete a pull request when it’s needed. You just need to set the settings in your repo settings. Navigate to your repo’s security setting, select the group and role your super users belong to and set the bypass policies’s settings.


CI/CD/CT at Enterprise Scale – Categorize Your Tests

Photo by: moren hsu

When it comes to enterprise software, you usually have to deal with huge number of tests (I hope that’s the case for you!). One of the biggest challenges in having an efficient CI/CD/CT pipeline is to have a small set of tests to run to make the integration and deployment faster. One of the techniques to reduce the testing time is to limit the testing scope. I previously post a blog about Test Impact Analysis here which is one of the most reliable ways to trim the testing scope but also by categorizing your tests you can achieve small test surface even in development environment. Remember if you expect developers to run ten thousand tests before committing the code that will never happen!

Using TestCategory lets you create a group of tests using any arbitrary system you want (i.e. Domain, Scenarios etc.). This gives you an opportunity to create and run related tests together to facilitate the testing in CI/CD/CT pipeline. The only thing you need to do is to annotate your tests with TestCategory attribute.

public void GetAllAircraft()

You can even assign a test to multiple category. If you want to filter your tests in Visual Studio by category you need to switch to list view in Test Explorer and then filter tests using Group By toggle.

CI/CD/CT at Enterprise Scale – Add label to Pull Requests

As a best practice, when we want to merge  a topic or feature branch back into the main branch we create a pull request. Developers sometimes may need to communicate some extra information with reviewers or bring their attention to specific details. In VSTS, labels provide a new way of communication for pull request.

You can tag your pull requests with labels like “Hotfix” or “Dependency” or even “DON’T MERGE” and draw reviewer attention to an important details.  Nowadays, many teams communicate things like this during the stand-ups or even through communication tools like Slack or Microsoft Team. However, when it comes to enterprise level app and big teams, traditional ways are not really practical because communications can easily get lost between team members.

CI/CD/CT at Enterprise Scale – Add Continuous Testing to Pull Requests

As a good practice we always branch out our feature branch from the main branch (E.g. master) and when we are done with the development, we send a pull request to merge changes back into the main branch.

Due to the fact that main branch always evolving and going ahead with changes from other branches, when it comes to merging back a feature branch to the main branch, changes could break the code. While we expect team members to update their feature branches with latest changes from the main branch before sending a pull request to reduce the risk of broken code, this is not working well at all the time.

This is why introducing continues testing into the pull request process removes the risk of broken code. This simply can be implemented in VSTS by adding a Build Validation as part of Branch Policies.

Navigate to list of branches, find your main branch and then select “Branch Policies”.

On the policies screen, click on Build Validations and add your build pipeline to the branch policies. Continuous test can be one of the steps in your build pipeline. You can make this policy Required or Optional.

You can also enable Test Impact Analysis (TIA) to reduce the testing and respectively building time.

You can fine more information about TIA here.

CI/CD/CT at Enterprise Scale – Enable Test Impact Analysis

As part of your CI/CD/CT pipeline you want to integrate, deploy and tests your application couple of times a day. When it comes to enterprise level application, we usually have a huge set of tests which makes it very hard and time consuming (if not impossible) to run every time as part of our continuous testing.

Test Impact Analysis (TIA) is a technique to determine the impacted tests for a given set of changes. Therefore,  you don’t need to run all of the tests every time you want to build and deploy a new version.

TIA is just a click away in VSTS – you can easily enable TIA as part of your CI/CD/CT pipeline to dramatically reduce the time needed to run the tests.

You just need to enable “Run only impacted tests” in your version 2.* and above build’s test step.