When it comes to enterprise software, you usually have to deal with huge number of tests (I hope that’s the case for you!). One of the biggest challenges in having an efficient CI/CD/CT pipeline is to have a small set of tests to run to make the integration and deployment faster. One of the techniques to reduce the testing time is to limit the testing scope. I previously post a blog about Test Impact Analysis here which is one of the most reliable ways to trim the testing scope but also by categorizing your tests you can achieve small test surface even in development environment. Remember if you expect developers to run ten thousand tests before committing the code that will never happen!
Using TestCategory lets you create a group of tests using any arbitrary system you want (i.e. Domain, Scenarios etc.). This gives you an opportunity to create and run related tests together to facilitate the testing in CI/CD/CT pipeline. The only thing you need to do is to annotate your tests with TestCategory attribute.
public void GetAllAircraft()
You can even assign a test to multiple category. If you want to filter your tests in Visual Studio by category you need to switch to list view in Test Explorer and then filter tests using Group By toggle.
As a best practice, when we want to merge a topic or feature branch back into the main branch we create a pull request. Developers sometimes may need to communicate some extra information with reviewers or bring their attention to specific details. In VSTS, labels provide a new way of communication for pull request.
You can tag your pull requests with labels like “Hotfix” or “Dependency” or even “DON’T MERGE” and draw reviewer attention to an important details. Nowadays, many teams communicate things like this during the stand-ups or even through communication tools like Slack or Microsoft Team. However, when it comes to enterprise level app and big teams, traditional ways are not really practical because communications can easily get lost between team members.
As a good practice we always branch out our feature branch from the main branch (E.g. master) and when we are done with the development, we send a pull request to merge changes back into the main branch.
Due to the fact that main branch always evolving and going ahead with changes from other branches, when it comes to merging back a feature branch to the main branch, changes could break the code. While we expect team members to update their feature branches with latest changes from the main branch before sending a pull request to reduce the risk of broken code, this is not working well at all the time.
This is why introducing continues testing into the pull request process removes the risk of broken code. This simply can be implemented in VSTS by adding a Build Validation as part of Branch Policies.
Navigate to list of branches, find your main branch and then select “Branch Policies”.
On the policies screen, click on Build Validations and add your build pipeline to the branch policies. Continuous test can be one of the steps in your build pipeline. You can make this policy Required or Optional.
You can also enable Test Impact Analysis (TIA) to reduce the testing and respectively building time.
As part of your CI/CD/CT pipeline you want to integrate, deploy and tests your application couple of times a day. When it comes to enterprise level application, we usually have a huge set of tests which makes it very hard and time consuming (if not impossible) to run every time as part of our continuous testing.
Test Impact Analysis (TIA) is a technique to determine the impacted tests for a given set of changes. Therefore, you don’t need to run all of the tests every time you want to build and deploy a new version.
TIA is just a click away in VSTS – you can easily enable TIA as part of your CI/CD/CT pipeline to dramatically reduce the time needed to run the tests.
You just need to enable “Run only impacted tests” in your version 2.* and above build’s test step.