If you have read my introduction to xUnit.NET, you know that tests are executed sequentially inside a collection. By default, a collection is created per each class. So, all tests in a class are executed sequentially. You may want to execute all tests in parallel to reduce the execution time. A workaround is to create one class per test, but this is not very practical. xUnit 3 should provide a way to execute all tests in parallel (GitHub issue), but this version is still in alpha.
This post is based on the code suggested by Travis Mortimer on GitHub. I've improved it a bit to make it more readable and to fix a bug with non-serializable [Theory].
To parallelize all test cases, you just need to add a NuGet package to your project:
csproj (MSBuild project file)
<Project>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.0.0" />
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.3" />
<PackageReference Include="Meziantou.Xunit.ParallelTestFramework" Version="1.0.0" />
</ItemGroup>
</Project>
#How does it work?
xUnit is very customizable. You can change the default behavior of the framework by implementing a few interfaces. To change the way tests are executed, you can override TestFramework.RunTestCases to change the list of test cases before the framework executes them.
In our case, the strategy is to reassign all test cases to a new collection, and then let xUnit execute the tests. As all tests are in a different collection, xUnit will execute them in parallel. That is that simple!
C#
// Inspired from https://github.com/xunit/xunit/issues/1986#issuecomment-831322722 by Travis Mortimer
namespace Xunit.Custom;
internal sealed class ParallelTestFramework : XunitTestFramework
{
public ParallelTestFramework(IMessageSink diagnosticMessageSink)
: base(diagnosticMessageSink)
{
}
protected override ITestFrameworkExecutor CreateExecutor(AssemblyName assemblyName)
{
return new CustomTestExecutor(assemblyName, SourceInformationProvider, DiagnosticMessageSink);
}
private sealed class CustomTestExecutor : XunitTestFrameworkExecutor
{
public CustomTestExecutor(AssemblyName assemblyName, ISourceInformationProvider sourceInformationProvider, IMessageSink diagnosticMessageSink)
: base(assemblyName, sourceInformationProvider, diagnosticMessageSink)
{
}
protected override async void RunTestCases(IEnumerable<IXunitTestCase> testCases, IMessageSink executionMessageSink, ITestFrameworkExecutionOptions executionOptions)
{
var newTestCases = SetUpTestCaseParallelization(testCases);
using var assemblyRunner = new XunitTestAssemblyRunner(TestAssembly, newTestCases, DiagnosticMessageSink, executionMessageSink, executionOptions);
await assemblyRunner.RunAsync();
}
/// <summary>
/// By default, all test cases in a test class share the same collection instance which ensures they run synchronously.
/// By providing a unique test collection instance to every test case in a test class you can make them all run in parallel.
/// </summary>
private IEnumerable<IXunitTestCase> SetUpTestCaseParallelization(IEnumerable<IXunitTestCase> testCases)
{
var result = new List<IXunitTestCase>();
foreach (var testCase in testCases)
{
var oldTestMethod = testCase.TestMethod;
var oldTestClass = oldTestMethod.TestClass;
var oldTestCollection = oldTestMethod.TestClass.TestCollection;
// If the collection is explicitly set, don't try to parallelize test execution
if (oldTestCollection.CollectionDefinition != null || oldTestClass.Class.GetCustomAttributes(typeof(CollectionAttribute)).Any())
{
result.Add(testCase);
continue;
}
// Create a new collection with a unique id for the test case.
var newTestCollection =
new TestCollection(
oldTestCollection.TestAssembly,
oldTestCollection.CollectionDefinition,
displayName: $"{oldTestCollection.DisplayName} {oldTestCollection.UniqueID}");
newTestCollection.UniqueID = Guid.NewGuid();
// Duplicate the test and assign it to the new collection
var newTestClass = new TestClass(newTestCollection, oldTestClass.Class);
var newTestMethod = new TestMethod(newTestClass, oldTestMethod.Method);
switch (testCase)
{
// Used by Theory having DisableDiscoveryEnumeration or non-serializable data
case XunitTheoryTestCase xunitTheoryTestCase:
result.Add(new XunitTheoryTestCase(
DiagnosticMessageSink,
GetTestMethodDisplay(xunitTheoryTestCase),
GetTestMethodDisplayOptions(xunitTheoryTestCase),
newTestMethod));
break;
// Used by all other tests
case XunitTestCase xunitTestCase:
result.Add(new XunitTestCase(
DiagnosticMessageSink,
GetTestMethodDisplay(xunitTestCase),
GetTestMethodDisplayOptions(xunitTestCase),
newTestMethod,
xunitTestCase.TestMethodArguments));
break;
// TODO If you use custom attribute, you may need to add cases here
default:
throw new ArgumentOutOfRangeException("Test case " + testCase.GetType() + " not supported");
}
}
return result;
static TestMethodDisplay GetTestMethodDisplay(TestMethodTestCase testCase)
{
return (TestMethodDisplay)typeof(TestMethodTestCase)
.GetProperty("DefaultMethodDisplay", BindingFlags.Instance | BindingFlags.NonPublic)!
.GetValue(testCase)!;
}
static TestMethodDisplayOptions GetTestMethodDisplayOptions(TestMethodTestCase testCase)
{
return (TestMethodDisplayOptions)typeof(TestMethodTestCase)
.GetProperty("DefaultMethodDisplayOptions", BindingFlags.Instance | BindingFlags.NonPublic)!
.GetValue(testCase)!;
}
}
}
}
Now you have to instruct xUnit to use the custom TestFramework by using TestFrameworkAttribute. You can use the following attribute in any file of the test project.
C#
[assembly: Xunit.TestFramework(typeName: "Xunit.Custom.ParallelTestFramework",
// TODO Use the Assembly Name of the project containing
// ParallelTestFramework instead of MyTestProject1
assemblyName: "MyTestProject1")]
The NuGet package Meziantou.Xunit.ParallelTestFramework contains the source code of ParallelTestFramework and add the assembly attribute.
#Additional resources
Do you have a question or a suggestion about this post? Contact me!
