MSTest v2: Execute tests in parallel

This post is part of the series 'MSTest v2'. Be sure to check out the rest of the blog posts of the series!

• MSTest v2: Setup a test project and run tests
• MSTest v2: Exploring asserts
• MSTest v2: Data tests
• MSTest v2: Test lifecycle attributes
• MSTest v2: Create new asserts
• MSTest v2: Customize test execution
• MSTest v2: Execute tests in parallel (this post)
• MSTest v2: Testing against multiple frameworks

By default, the MSTest runner executes the tests of an assembly sequentially. If your tests are well isolated you can run them in parallel to reduce the execution time. Since version 1.3, MSTest v2 provides this functionality, but you have to opt-in.

To opt-in for parallel execution, you just have to set the [Parallelize] attribute in the assembly:

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

The attribute has 2 properties: Workers et Scope.

• Workers: The number of threads to run the tests. Set it to 0 to use the number of cores of your computer.
• Scope: Determine if the runner must parallelize tests at the method or class level. MethodLevel will run all tests in parallel. ClassLevel will run all test classes in parallel, but tests in a class are run sequentially. You should use ClassLevel if the tests within classes have interdependencies.

If you have multiple test assemblies you want to parallelize you can create a file named .runsettings at the root of the solution:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <MSTest>
    <Parallelize>
      <Workers>8</Workers>
      <Scope>MethodLevel</Scope>
    </Parallelize>
  </MSTest>
</RunSettings>

If you want to get the maximum performance, you should use Workers = 0 and Scope = ExecutionScope.MethodLevel. Note that the scope is MethodLevel. A method can contains multiple tests if you use data test. Each row of a data test is executed sequentially.

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

namespace UnitTestProject1
{
    [TestClass]
    public class TestClass1
    {
        [TestMethod]
        public void Test1() => Thread.Sleep(5000);

        [TestMethod]
        public void Test2() => Thread.Sleep(5000);

        [DataTestMethod]
        [DataRow(1)]
        [DataRow(2)]
        [DataRow(3)]
        public void Test3(int i) => Thread.Sleep(5000);
    }
}

The 3 test methods are executed in parallel, but the 3 tests of Test3 are run sequentially. So, the total execution time will be 15 seconds.

Enabling parallelization is opt-in. Once it's enabled, you can opt-out for a specific test or class using [DoNotParallelize].

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

namespace UnitTestProject1
{
    [TestClass]
    public class TestClass1
    {
        [TestMethod]
        [DoNotParallelize]
        public void Test1() => Thread.Sleep(5000);

        [TestMethod]
        public void Test2() => Thread.Sleep(5000);

        [TestMethod]
        public void Test3() => Thread.Sleep(5000);
    }
}

The runner will execute Test2 and Test3 in parallel. Then, it will execute Test1.

#Conclusion

In most cases, you can reduce the time required to run your tests by adding one attribute to your project 😃

Do you have a question or a suggestion about this post? Contact me!