All the tests are created using @Test annotation is static test cases. In this article you will see how to create Junit 5 dynamic tests with @TestFactory, and you will see several examples of Junit 5 Dynamic tests according to test lifecycle, timeouts, nested tests, parallel execution and test execution order.
Technologies used in following examples :
- Junit 5.5.2
- Maven 3
- Java 8
- Spring Tool Suite 3.9.8
1. Junit 5 Dynamic tests
A dynamic test is a test that generated at runtime by factory method using @TestFactory annotation. The method marked @TestFactory is not a test case, rather it’s a factory for test cases. Following is a simple dynamic test case example.
public class Junit5_Dynamic_Tests {
// Static test 1
@Test
void test_Add() {
assertEquals(5, MathUtil.add(3, 2));
}
// This method produces Dynamic test cases
@TestFactory
Collection<DynamicTest> dynamicTestsFromCollection() {
return Arrays.asList(
dynamicTest("1st dynamic test", () -> assertTrue(MathUtil.isPrime(13))),
dynamicTest("2nd dynamic test", () -> assertEquals(5, MathUtil.devide(25, 5)))
);
}
// Static test 2
@Test
void test_Devide() {
assertEquals(5, MathUtil.devide(25, 5));
}
}
Output :
2. Rules to create Junit 5 Dynamic Tests
@TestFactorymethods must not beprivateorstatic.@TestFactorymethods must returnStream,Collection,Iterable,Iterator, or array ofDynamicNodeinstances.- Also can return sub classes of
DynamicNode, they areDynamicContainerandDynamicTest.
Here is another example.
public class Junit5_Dynamic_Another_Test {
@TestFactory
DynamicTest dynamicTest() {
return DynamicTest.dynamicTest("Single dynamic test",
() -> assertTrue(MathUtil.isPrime(13)));
}
@TestFactory
DynamicContainer dynamicTestsFromStream() {
return DynamicContainer.dynamicContainer("DynamicContainer",
Stream.of(
DynamicTest.dynamicTest("1st container test",
() -> assertTrue(MathUtil.isPrime(13))),
DynamicTest.dynamicTest("2nd container test",
() -> assertEquals(5, MathUtil.devide(25, 5))
)));
}
// This method produces Dynamic test cases
@TestFactory
Stream<DynamicNode> dynamicTestsFromCollection() {
return Stream.of(7, 13)
.map(number -> DynamicContainer.dynamicContainer("Prime or Odd Test"+number,
Stream.of(
DynamicTest.dynamicTest("is number "+number+" prime?",
() -> assertTrue(MathUtil.isPrime(number))),
DynamicTest.dynamicTest("is number "+number+" odd?",
() -> assertFalse(MathUtil.isEven(number))
))));
}
}
Output :
3. Lifecycle methods for dynamic tests
In the dynamic test, @BeforeEach and @AfterEach lifecycle methods are executed for the each @TestFactory method but not for each dynamic test. To know about test instance life cycle and licfecycle methods you refer article Junit 5 Test Instance lifecycle.
public class Junit5_DynamicTest_Lifecycle_Test {
@BeforeEach
void beforeEach(TestInfo info) {
System.out.println("Before execute "+info.getTestMethod().get().getName());
}
@TestFactory
Collection<DynamicTest> dynamicTestsFromCollection() {
return Arrays.asList(
dynamicTest("1st dynamic test", () -> System.out.println("Dynamic test 1")),
dynamicTest("2nd dynamic test", () -> System.out.println("Dynamic test 2")));
}
@AfterEach
void afterEach(TestInfo info) {
System.out.println("After execute "+info.getTestMethod().get().getName());
}
}
Output :
Before execute dynamicTestsFromCollection Dynamic test 1 Dynamic test 2 After execute dynamicTestsFromCollection
4. hierarchy or nesting dynamic tests example
You can nest or represent hierarchy of dynamic tests using dynamic containers of containers. For nested test cases you can refer article Junit 5 nested tests
public class Dynamic_Nested_Tests {
@TestFactory
Collection<DynamicContainer> dynamicContainerFromCollection() {
return asList(
dynamicContainer(
"1st root container",
asList(
dynamicContainer(
"1st Dynamic Container",
asList(
dynamicTest("1A dynamic test",
() -> assertTrue(MathUtil.isPrime(13))),
dynamicTest("1B dynamic test",
() -> assertEquals(5, MathUtil.devide(25, 5)))
)), // end of 1st container
dynamicContainer(
"2nd Dynamic Container",
asList(
dynamicTest("2A dynamic test",
() -> assertTrue(MathUtil.isPrime(13))),
dynamicTest("2B dynamic test",
() -> assertEquals(5, MathUtil.devide(25, 5)))
)) // end of 2nd container
)
), // End of 1st root container
dynamicContainer(
"2nd root container",
asList(
dynamicContainer(
"3rd Dynamic Container",
asList(
dynamicTest("3A dynamic test",
() -> assertTrue(MathUtil.isPrime(13))),
dynamicTest("3B dynamic test",
() -> assertEquals(5, MathUtil.devide(25, 5)))
)), // end of 3rd container
dynamicContainer(
"4th Dynamic Container",
asList(
dynamicTest("4A dynamic test",
() -> assertTrue(MathUtil.isPrime(13))),
dynamicTest("4B dynamic test",
() -> assertEquals(5, MathUtil.devide(25, 5)))
)) // end of 4th container
)
) // End of 2nd root container
); // End of complete List
}
}
Output :
5. Timeout in Dynamic tests
Declaring @Timeout on a @TestFactory method checks that the factory method returns within the specified duration but does not verify the execution time of each individual DynamicTest generated by the factory. For that purpose we can use assertTimeout() or assertTimeoutPreemptively() . You can see about timeout tests in article Junit 5 Timeout tests and @TestFactory
public class Junit5_DynamicTests_Timeout_Test {
@TestFactory
Collection<DynamicTest> test_dynamicTests_AssertTimeouts() {
return Arrays.asList(
dynamicTest("1st dynamic test", () -> {
assertTimeout(Duration.ofSeconds(5), () -> {
TimeUnit.SECONDS.sleep(10);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("Dynamic Test 3");
});
}),
dynamicTest("2nd dynamic test", () -> {
assertTimeoutPreemptively(Duration.ofSeconds(5), () -> {
TimeUnit.SECONDS.sleep(10);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("Dynamic Test 4");
});
}));
}
}
6. Execution order in dynamic tests
The method order for dynamic tests is not depends on @TestMethodOrder declared on top level test class. to control order of dynamic tests we can implement a custom sort. See the following example, that demonstrates how to control execution order in descending order of display names.
@TestMethodOrder(MethodOrderer.Alphanumeric.class)
public class Junit5_DynamicTests_Order_Test {
@TestFactory
Collection<DynamicTest> test_dynamicTests_AssertTimeouts() {
List<DynamicTest> dynamicTests = Arrays.asList(
dynamicTest("1st dynamic test", () -> {
assertTrue(MathUtil.isPrime(13));
System.out.println("=> 1st dynamic test");
}),
dynamicTest("2nd dynamic test", () -> {
assertEquals(5, MathUtil.devide(25, 5));
System.out.println("=> 2nd dynamic test");
}),
dynamicTest("3rd dynamic test", () -> {
assertEquals(12, MathUtil.add(7, 5));
System.out.println("=> 3rd dynamic test");
})
);
sortDynamicTests(dynamicTests);
return dynamicTests;
}
static void sortDynamicTests(List<DynamicTest> dynamicTests) {
dynamicTests.sort((DynamicTest d1, DynamicTest d2) ->
d2.getDisplayName().compareTo(d1.getDisplayName()));
}
}
Output :
=> 3rd dynamic test => 2nd dynamic test => 1st dynamic test
7. Parallel Test execution in Dynamic tests example
To enable parallel testing in Junit platform, include following properties in junit-platform.properties in src/test/resources folder.
junit.jupiter.execution.parallel.enabled=true junit.jupiter.execution.parallel.config.strategy=dynamic
You can apply @Execution(ExecutionMode.CONCURRENT) at top level class for the concurrent execution of dynamic tests Or you can apply to the @TestFactory method. Let’s have look into following example for the concurrent execution behavior.
//@Execution(ExecutionMode.CONCURRENT)
public class Junit5_DynamiTests_Parallel_Test {
@Execution(ExecutionMode.CONCURRENT)
@TestFactory
Collection<DynamicTest> test_parallel_dynamictests1() {
return Arrays.asList(
dynamicTest("1st dynamic test", () -> {
assertTrue(MathUtil.isPrime(13));
System.out.println(Thread.currentThread().getName()+" => 1st dynamic test");
}),
dynamicTest("2nd dynamic test", () -> {
assertEquals(5, MathUtil.devide(25, 5));
System.out.println(Thread.currentThread().getName()+" => 2nd dynamic test");
}),
dynamicTest("3rd dynamic test", () -> {
assertEquals(12, MathUtil.add(7, 5));
System.out.println(Thread.currentThread().getName()+" => 3rd dynamic test");
})
);
}
@Execution(ExecutionMode.SAME_THREAD)
@TestFactory
Collection<DynamicTest> test_parallel_dynamictests2() {
return Arrays.asList(
dynamicTest("4th dynamic test", () -> {
assertTrue(MathUtil.isPrime(13));
System.out.println(Thread.currentThread().getName()+" => 4th dynamic test");
}),
dynamicTest("5th dynamic test", () -> {
assertEquals(5, MathUtil.devide(25, 5));
System.out.println(Thread.currentThread().getName()+" => 5th dynamic test");
}),
dynamicTest("6th dynamic test", () -> {
assertEquals(12, MathUtil.add(7, 5));
System.out.println(Thread.currentThread().getName()+" => 6th dynamic test");
})
);
}
}
Output :
The dynamic tests generated by test_parallel_dynamictests2() are executed with in same thread worker-3. The other method test_parallel_dynamictests1() generated dynamic tests run by different thread workers concurrently.
ForkJoinPool-1-worker-3 => 4th dynamic test ForkJoinPool-1-worker-7 => 2nd dynamic test ForkJoinPool-1-worker-1 => 3rd dynamic test ForkJoinPool-1-worker-5 => 1st dynamic test ForkJoinPool-1-worker-3 => 5th dynamic test ForkJoinPool-1-worker-3 => 6th dynamic test
You remove all @Execution(..) annotations from above example and run the test, you will see following output.
ForkJoinPool-1-worker-3 => 1st dynamic test ForkJoinPool-1-worker-3 => 2nd dynamic test ForkJoinPool-1-worker-3 => 3rd dynamic test ForkJoinPool-1-worker-3 => 4th dynamic test ForkJoinPool-1-worker-3 => 5th dynamic test ForkJoinPool-1-worker-3 => 6th dynamic test
8. Parameterized dynamic tests
To create parameterized tests, we do something where we simply looped over the data and called the test method with it or we can inject a stream or collection or parameter to @TestFactory method.
public class Junit5_Dynamic_Parameterized_Test {
@TestFactory
Collection<DynamicTest> dynamicTestsFromCollection() {
List<DynamicTest> dynamicTests = new ArrayList<DynamicTest>();
Arrays.asList(7, 13, 17)
.forEach(number -> {
dynamicTests.add(
dynamicTest("is number "+number+" prime?",
() -> assertTrue(MathUtil.isPrime(number))));
dynamicTests.add(
dynamicTest("is number "+number+" even?",
() -> assertFalse(MathUtil.isEven(number))));
});
return dynamicTests;
}
}
Output :
