In this article we will see what are the Junit 5 timemouts, how to use @Timeout in writing test cases, how to use global timeouts and how to disable timeouts globally with several examples.
1. Junit 5 timeouts
Junit 5 timeouts allows to declare that a test, test factory, test template, or lifecycle method should fail if its execution time exceeds a given duration and will get java.util.concurrent.TimeoutException as results. The time unit for the duration defaults to seconds but is configurable.
Following are the several ways Junit 5 supports for the timeouts.
- Using
@Timeoutannotation. - Using
assertTimeout()orassertTimeoutPreemptively(). - Using configuration parameters to specify global timeouts .
1.1. Using @Timeout annotation
The following example shows how @Timeout is applied test methods and test class and test lifecycle.
Example 1 :
If @Timeout declared at class level, same timeouts applied for test methods, nested tests and test factories, but not applied timeout for @BeforeEach, @BeforeAll, @AfterEach, @AfterAll life cycle callback methods.
@Timeout(7) // each test fails if execution time exceeds 7 seconds
public class Junit5_Timeout_Annotation_ClassLevel_Test {
@BeforeEach
void setUp() throws Exception {
TimeUnit.SECONDS.sleep(10); // waits 10 seconds
System.out.println("@BeforeEach");
System.out.println("-------------------------");
}
@BeforeAll
static void setUpBeforeAllTests() throws Exception {
TimeUnit.SECONDS.sleep(10);
System.out.println("@BeforeAll");
}
@Test
void test_Add() throws Exception {
TimeUnit.SECONDS.sleep(10);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("test_Add()");
}
@Test
void test_Multiply() throws Exception {
TimeUnit.SECONDS.sleep(5); // waits 5 seconds
System.out.println("test_Multiply()");
}
}
Output :
@BeforeAll @BeforeEach ------------------------- @BeforeEach ------------------------- test_Multiply()
Results in eclipse :
Example 2 :
If @Timeout declared at @BeforeAll and exceeds timeout, execution will be aborted, skips the rest of test cases.
public class Junit5_Timeout_Annotation_Test {
@BeforeAll
@Timeout(4) // skip all tests if this method execution time exceeds 4 seconds
static void setUpBeforeAllTests() throws Exception {
TimeUnit.SECONDS.sleep(3);
System.out.println("@BeforeAll");
}
@BeforeEach
@Timeout(4) // fail all tests if this method execution time exceeds 4 seconds
void setUp() throws Exception {
TimeUnit.SECONDS.sleep(3);
System.out.println("@BeforeEach");
}
@Test
@Timeout(9) // this test fails if execution time exceeds 9 seconds
void test_Add() throws Exception {
TimeUnit.SECONDS.sleep(10); // waits 10 seconds
System.out.println("test_Add()");
}
@Test
@Timeout(9) // this test fails if execution time exceeds 9 seconds
void test_Multiply() throws Exception {
TimeUnit.SECONDS.sleep(8); // waits 8 seconds
System.out.println("test_Multiply()");
}
@Test
@Timeout(value = 900, unit = TimeUnit.MILLISECONDS)
void test_isPrime() throws Exception {
// fails if execution time exceeds 900 milli seconds
TimeUnit.MILLISECONDS.sleep(1000); // waits 1000 milli seconds
assertEquals(10, MathUtil.multiple(5, 2));
System.out.println("test_isPrime()");
}
}
Output :
@BeforeAll @BeforeEach @BeforeEach test_Multiply() @BeforeEach
1.2. assertTimeout() or assertTimeoutPreemptively()
1.2.1. assertTimeout() – Test method will be failed if not executed within time, but assertTimeout() completes its execution process.
1.2.2. assertTimeoutPreemptively() – Test method will be failed if not executed within time, assertTimeoutPreemptively() will be aborted execution process. This is useful to test transaction management tests, to ensure that abort the transaction operation if it exceeds the time.
@Test
void test_Add() throws Exception {
assertTimeout(Duration.ofSeconds(5), () -> {
TimeUnit.SECONDS.sleep(4);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("test_Add()");
});
}
@Test
void test_Multiply() throws Exception {
// fails but completes the process, see output and run duration
assertTimeout(Duration.ofSeconds(5), () -> {
TimeUnit.SECONDS.sleep(6);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("test_Multiply()");
});
}
@Test
void test_assertTimeoutPreemptively() throws Exception {
// fails but abort the process, see output and run duration
assertTimeoutPreemptively(Duration.ofSeconds(5), () -> {
TimeUnit.SECONDS.sleep(6);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("test_assertTimeoutPreemptively()");
});
}
Output :
test_Add() test_Multiply()
Execution Time and status of each Test :
- test_Add() – Passed – 4.035 s
- test_Multiply() – Failed – 10.011 s
- test_assertTimeoutPreemptively() – Failed – 5.016 s
1.2.3. Declaring @Timeout on a @TestFactory method checks that the factory method returns within the specified duration but does not verify the execution time of each individual DynamicTest generated by the factory. We can use assertTimeout() or assertTimeoutPreemptively() for that purpose.
Junit 5 Timeout for Dynamic tests example :
@TestFactory
@Timeout(5)
Collection<DynamicTest> test_dynamicTestsTimeouts() {
return Arrays.asList(
dynamicTest("1st dynamic test", () -> {
TimeUnit.SECONDS.sleep(3);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("Dynamic Test 1");
}),
dynamicTest("2nd dynamic test", () -> {
TimeUnit.SECONDS.sleep(2);
assertEquals(6, MathUtil.multiple(3, 2));
System.out.println("Dynamic Test 2");
}));
}
@TestFactory
Collection<DynamicTest> test_dynamicTests_AssertTimeouts() {
return Arrays.asList(
dynamicTest("3rd dynamic test", () -> {
assertTimeout(Duration.ofSeconds(5), () -> {
TimeUnit.SECONDS.sleep(10);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("Dynamic Test 3");
});
}),
dynamicTest("4th dynamic test", () -> {
assertTimeoutPreemptively(Duration.ofSeconds(5), () -> {
TimeUnit.SECONDS.sleep(10);
assertEquals(5, MathUtil.add(3, 2));
System.out.println("Dynamic Test 4");
});
}));
}
Output :
Dynamic Test 1 Dynamic Test 2 Dynamic Test 3
Execution Time and status of each Test :
- 1st dynamic test – Passed – 3.006 s
- 2nd dynamic test – Passed – 2.004 s
- 3rd dynamic test – Failed – 10.004 s
- 4th dynamic test – Failed – 5.003 s
1.3. Junit 5 global timeouts
Configuration parameters can be used to specify global tiemouts. For example to specify timeouts for all the @BeforeEach methods in entire test application set the value for configuration property junit.jupiter.execution.timeout.beforeeach.method.default
Following can be used to specify global timeouts. Values can be set in junit-platform.properties file under test/resources folder or can be set in maven sure-fire plugin configurationParameters properties or can set as System property value can be provide from maven command.
junit.jupiter.execution.timeout.default– Default timeout for all testable and lifecycle methods.junit.jupiter.execution.timeout.testable.method.default– Default timeout for all testable methods.junit.jupiter.execution.timeout.test.method.default– Default timeout for@Testmethods.junit.jupiter.execution.timeout.testtemplate.method.default– Default timeout for@TestTemplatemethods.junit.jupiter.execution.timeout.testfactory.method.default– Default timeout for@TestFactorymethods.junit.jupiter.execution.timeout.lifecycle.method.default– Default timeout for all lifecycle methods.junit.jupiter.execution.timeout.beforeall.method.default– Default timeout for@BeforeAllmethods.junit.jupiter.execution.timeout.beforeeach.method.default– Default timeout for@BeforeEachmethods.junit.jupiter.execution.timeout.afterall.method.default– Default timeout for@AfterAllmethods.junit.jupiter.execution.timeout.aftereach.method.default– Default timeout for@AfterEachmethods.
Example junit-platform.properties :
junit.jupiter.execution.timeout.beforeeach.method.default=200 ms junit.jupiter.execution.timeout.test.method.default=10 s
2. Disable @Timeout globally
When we are debugging test cases timeout may influence the results of test execution. For example assume that, when writing tests we set the timeout value 10 seconds to complete transactions related tests, but later it increased to 20 seconds. All the tests may impact and mark the test as failed although all assertions were met.
JUnit Jupiter supports the junit.jupiter.execution.timeout.mode configuration parameter to configure when timeouts are applied. There are three modes.
enableddisableddisabled_on_debug
Example junit-platform.properties :
junit.jupiter.execution.timeout.mode=disabled
Conclusion
In this article we have seen what are the Junit 5 timemouts, how to use @Timeout in writing test cases, how to use global timeouts and how to disable timeouts globally with several examples. Declarative timeouts are an experimental feature, but this is very useful feature to handle timeouts globally.
