All the tests are created using @Test annotation is static test cases. In this article you will see how to create Junit 5 dynamic tests with @TestFactory, and you will see several examples of Junit 5 Dynamic tests according to test lifecycle, timeouts, nested tests, parallel execution and test execution order.
Technologies used in following examples :
- Junit 5.5.2
- Maven 3
- Java 8
- Spring Tool Suite 3.9.8
1. Junit 5 Dynamic tests
A dynamic test is a test that generated at runtime by factory method using @TestFactory
annotation. The method marked @TestFactory
is not a test case, rather it’s a factory for test cases. Following is a simple dynamic test case example.
public class Junit5_Dynamic_Tests { // Static test 1 @Test void test_Add() { assertEquals(5, MathUtil.add(3, 2)); } // This method produces Dynamic test cases @TestFactory Collection<DynamicTest> dynamicTestsFromCollection() { return Arrays.asList( dynamicTest("1st dynamic test", () -> assertTrue(MathUtil.isPrime(13))), dynamicTest("2nd dynamic test", () -> assertEquals(5, MathUtil.devide(25, 5))) ); } // Static test 2 @Test void test_Devide() { assertEquals(5, MathUtil.devide(25, 5)); } }
Output :
2. Rules to create Junit 5 Dynamic Tests
@TestFactory
methods must not beprivate
orstatic
.@TestFactory
methods must returnStream
,Collection
,Iterable
,Iterator
, or array ofDynamicNode
instances.- Also can return sub classes of
DynamicNode
, they areDynamicContainer
andDynamicTest
.
Here is another example.
public class Junit5_Dynamic_Another_Test { @TestFactory DynamicTest dynamicTest() { return DynamicTest.dynamicTest("Single dynamic test", () -> assertTrue(MathUtil.isPrime(13))); } @TestFactory DynamicContainer dynamicTestsFromStream() { return DynamicContainer.dynamicContainer("DynamicContainer", Stream.of( DynamicTest.dynamicTest("1st container test", () -> assertTrue(MathUtil.isPrime(13))), DynamicTest.dynamicTest("2nd container test", () -> assertEquals(5, MathUtil.devide(25, 5)) ))); } // This method produces Dynamic test cases @TestFactory Stream<DynamicNode> dynamicTestsFromCollection() { return Stream.of(7, 13) .map(number -> DynamicContainer.dynamicContainer("Prime or Odd Test"+number, Stream.of( DynamicTest.dynamicTest("is number "+number+" prime?", () -> assertTrue(MathUtil.isPrime(number))), DynamicTest.dynamicTest("is number "+number+" odd?", () -> assertFalse(MathUtil.isEven(number)) )))); } }
Output :
3. Lifecycle methods for dynamic tests
In the dynamic test, @BeforeEach
and @AfterEach
lifecycle methods are executed for the each @TestFactory
method but not for each dynamic test. To know about test instance life cycle and licfecycle methods you refer article Junit 5 Test Instance lifecycle.
public class Junit5_DynamicTest_Lifecycle_Test { @BeforeEach void beforeEach(TestInfo info) { System.out.println("Before execute "+info.getTestMethod().get().getName()); } @TestFactory Collection<DynamicTest> dynamicTestsFromCollection() { return Arrays.asList( dynamicTest("1st dynamic test", () -> System.out.println("Dynamic test 1")), dynamicTest("2nd dynamic test", () -> System.out.println("Dynamic test 2"))); } @AfterEach void afterEach(TestInfo info) { System.out.println("After execute "+info.getTestMethod().get().getName()); } }
Output :
Before execute dynamicTestsFromCollection Dynamic test 1 Dynamic test 2 After execute dynamicTestsFromCollection
4. hierarchy or nesting dynamic tests example
You can nest or represent hierarchy of dynamic tests using dynamic containers of containers. For nested test cases you can refer article Junit 5 nested tests
public class Dynamic_Nested_Tests { @TestFactory Collection<DynamicContainer> dynamicContainerFromCollection() { return asList( dynamicContainer( "1st root container", asList( dynamicContainer( "1st Dynamic Container", asList( dynamicTest("1A dynamic test", () -> assertTrue(MathUtil.isPrime(13))), dynamicTest("1B dynamic test", () -> assertEquals(5, MathUtil.devide(25, 5))) )), // end of 1st container dynamicContainer( "2nd Dynamic Container", asList( dynamicTest("2A dynamic test", () -> assertTrue(MathUtil.isPrime(13))), dynamicTest("2B dynamic test", () -> assertEquals(5, MathUtil.devide(25, 5))) )) // end of 2nd container ) ), // End of 1st root container dynamicContainer( "2nd root container", asList( dynamicContainer( "3rd Dynamic Container", asList( dynamicTest("3A dynamic test", () -> assertTrue(MathUtil.isPrime(13))), dynamicTest("3B dynamic test", () -> assertEquals(5, MathUtil.devide(25, 5))) )), // end of 3rd container dynamicContainer( "4th Dynamic Container", asList( dynamicTest("4A dynamic test", () -> assertTrue(MathUtil.isPrime(13))), dynamicTest("4B dynamic test", () -> assertEquals(5, MathUtil.devide(25, 5))) )) // end of 4th container ) ) // End of 2nd root container ); // End of complete List } }
Output :
5. Timeout in Dynamic tests
Declaring @Timeout
on a @TestFactory
method checks that the factory method returns within the specified duration but does not verify the execution time of each individual DynamicTest
generated by the factory. For that purpose we can use assertTimeout()
or assertTimeoutPreemptively()
. You can see about timeout tests in article Junit 5 Timeout tests and @TestFactory
public class Junit5_DynamicTests_Timeout_Test { @TestFactory Collection<DynamicTest> test_dynamicTests_AssertTimeouts() { return Arrays.asList( dynamicTest("1st dynamic test", () -> { assertTimeout(Duration.ofSeconds(5), () -> { TimeUnit.SECONDS.sleep(10); assertEquals(5, MathUtil.add(3, 2)); System.out.println("Dynamic Test 3"); }); }), dynamicTest("2nd dynamic test", () -> { assertTimeoutPreemptively(Duration.ofSeconds(5), () -> { TimeUnit.SECONDS.sleep(10); assertEquals(5, MathUtil.add(3, 2)); System.out.println("Dynamic Test 4"); }); })); } }
6. Execution order in dynamic tests
The method order for dynamic tests is not depends on @TestMethodOrder
declared on top level test class. to control order of dynamic tests we can implement a custom sort. See the following example, that demonstrates how to control execution order in descending order of display names.
@TestMethodOrder(MethodOrderer.Alphanumeric.class) public class Junit5_DynamicTests_Order_Test { @TestFactory Collection<DynamicTest> test_dynamicTests_AssertTimeouts() { List<DynamicTest> dynamicTests = Arrays.asList( dynamicTest("1st dynamic test", () -> { assertTrue(MathUtil.isPrime(13)); System.out.println("=> 1st dynamic test"); }), dynamicTest("2nd dynamic test", () -> { assertEquals(5, MathUtil.devide(25, 5)); System.out.println("=> 2nd dynamic test"); }), dynamicTest("3rd dynamic test", () -> { assertEquals(12, MathUtil.add(7, 5)); System.out.println("=> 3rd dynamic test"); }) ); sortDynamicTests(dynamicTests); return dynamicTests; } static void sortDynamicTests(List<DynamicTest> dynamicTests) { dynamicTests.sort((DynamicTest d1, DynamicTest d2) -> d2.getDisplayName().compareTo(d1.getDisplayName())); } }
Output :
=> 3rd dynamic test => 2nd dynamic test => 1st dynamic test
7. Parallel Test execution in Dynamic tests example
To enable parallel testing in Junit platform, include following properties in junit-platform.properties
in src/test/resources
folder.
junit.jupiter.execution.parallel.enabled=true junit.jupiter.execution.parallel.config.strategy=dynamic
You can apply @Execution(ExecutionMode.CONCURRENT)
at top level class for the concurrent execution of dynamic tests Or you can apply to the @TestFactory
method. Let’s have look into following example for the concurrent execution behavior.
//@Execution(ExecutionMode.CONCURRENT) public class Junit5_DynamiTests_Parallel_Test { @Execution(ExecutionMode.CONCURRENT) @TestFactory Collection<DynamicTest> test_parallel_dynamictests1() { return Arrays.asList( dynamicTest("1st dynamic test", () -> { assertTrue(MathUtil.isPrime(13)); System.out.println(Thread.currentThread().getName()+" => 1st dynamic test"); }), dynamicTest("2nd dynamic test", () -> { assertEquals(5, MathUtil.devide(25, 5)); System.out.println(Thread.currentThread().getName()+" => 2nd dynamic test"); }), dynamicTest("3rd dynamic test", () -> { assertEquals(12, MathUtil.add(7, 5)); System.out.println(Thread.currentThread().getName()+" => 3rd dynamic test"); }) ); } @Execution(ExecutionMode.SAME_THREAD) @TestFactory Collection<DynamicTest> test_parallel_dynamictests2() { return Arrays.asList( dynamicTest("4th dynamic test", () -> { assertTrue(MathUtil.isPrime(13)); System.out.println(Thread.currentThread().getName()+" => 4th dynamic test"); }), dynamicTest("5th dynamic test", () -> { assertEquals(5, MathUtil.devide(25, 5)); System.out.println(Thread.currentThread().getName()+" => 5th dynamic test"); }), dynamicTest("6th dynamic test", () -> { assertEquals(12, MathUtil.add(7, 5)); System.out.println(Thread.currentThread().getName()+" => 6th dynamic test"); }) ); } }
Output :
The dynamic tests generated by test_parallel_dynamictests2()
are executed with in same thread worker-3
. The other method test_parallel_dynamictests1()
generated dynamic tests run by different thread workers concurrently.
ForkJoinPool-1-worker-3 => 4th dynamic test ForkJoinPool-1-worker-7 => 2nd dynamic test ForkJoinPool-1-worker-1 => 3rd dynamic test ForkJoinPool-1-worker-5 => 1st dynamic test ForkJoinPool-1-worker-3 => 5th dynamic test ForkJoinPool-1-worker-3 => 6th dynamic test
You remove all @Execution(..)
annotations from above example and run the test, you will see following output.
ForkJoinPool-1-worker-3 => 1st dynamic test ForkJoinPool-1-worker-3 => 2nd dynamic test ForkJoinPool-1-worker-3 => 3rd dynamic test ForkJoinPool-1-worker-3 => 4th dynamic test ForkJoinPool-1-worker-3 => 5th dynamic test ForkJoinPool-1-worker-3 => 6th dynamic test
8. Parameterized dynamic tests
To create parameterized tests, we do something where we simply looped over the data and called the test method with it or we can inject a stream or collection or parameter to @TestFactory
method.
public class Junit5_Dynamic_Parameterized_Test { @TestFactory Collection<DynamicTest> dynamicTestsFromCollection() { List<DynamicTest> dynamicTests = new ArrayList<DynamicTest>(); Arrays.asList(7, 13, 17) .forEach(number -> { dynamicTests.add( dynamicTest("is number "+number+" prime?", () -> assertTrue(MathUtil.isPrime(number)))); dynamicTests.add( dynamicTest("is number "+number+" even?", () -> assertFalse(MathUtil.isEven(number)))); }); return dynamicTests; } }
Output :