In this article we will see what are the Junit 5 timemouts, how to use @Timeout
in writing test cases, how to use global timeouts and how to disable timeouts globally with several examples.
1. Junit 5 timeouts
Junit 5 timeouts allows to declare that a test, test factory, test template, or lifecycle method should fail if its execution time exceeds a given duration and will get java.util.concurrent.TimeoutException
as results. The time unit for the duration defaults to seconds but is configurable.
Following are the several ways Junit 5 supports for the timeouts.
- Using
@Timeout
annotation. - Using
assertTimeout()
orassertTimeoutPreemptively()
. - Using configuration parameters to specify global timeouts .
1.1. Using @Timeout annotation
The following example shows how @Timeout
is applied test methods and test class and test lifecycle.
Example 1 :
If @Timeout
declared at class level, same timeouts applied for test methods, nested tests and test factories, but not applied timeout for @BeforeEach
, @BeforeAll
, @AfterEach
, @AfterAll
life cycle callback methods.
@Timeout(7) // each test fails if execution time exceeds 7 seconds public class Junit5_Timeout_Annotation_ClassLevel_Test { @BeforeEach void setUp() throws Exception { TimeUnit.SECONDS.sleep(10); // waits 10 seconds System.out.println("@BeforeEach"); System.out.println("-------------------------"); } @BeforeAll static void setUpBeforeAllTests() throws Exception { TimeUnit.SECONDS.sleep(10); System.out.println("@BeforeAll"); } @Test void test_Add() throws Exception { TimeUnit.SECONDS.sleep(10); assertEquals(5, MathUtil.add(3, 2)); System.out.println("test_Add()"); } @Test void test_Multiply() throws Exception { TimeUnit.SECONDS.sleep(5); // waits 5 seconds System.out.println("test_Multiply()"); } }
Output :
@BeforeAll @BeforeEach ------------------------- @BeforeEach ------------------------- test_Multiply()
Results in eclipse :
Example 2 :
If @Timeout
declared at @BeforeAll
and exceeds timeout, execution will be aborted, skips the rest of test cases.
public class Junit5_Timeout_Annotation_Test { @BeforeAll @Timeout(4) // skip all tests if this method execution time exceeds 4 seconds static void setUpBeforeAllTests() throws Exception { TimeUnit.SECONDS.sleep(3); System.out.println("@BeforeAll"); } @BeforeEach @Timeout(4) // fail all tests if this method execution time exceeds 4 seconds void setUp() throws Exception { TimeUnit.SECONDS.sleep(3); System.out.println("@BeforeEach"); } @Test @Timeout(9) // this test fails if execution time exceeds 9 seconds void test_Add() throws Exception { TimeUnit.SECONDS.sleep(10); // waits 10 seconds System.out.println("test_Add()"); } @Test @Timeout(9) // this test fails if execution time exceeds 9 seconds void test_Multiply() throws Exception { TimeUnit.SECONDS.sleep(8); // waits 8 seconds System.out.println("test_Multiply()"); } @Test @Timeout(value = 900, unit = TimeUnit.MILLISECONDS) void test_isPrime() throws Exception { // fails if execution time exceeds 900 milli seconds TimeUnit.MILLISECONDS.sleep(1000); // waits 1000 milli seconds assertEquals(10, MathUtil.multiple(5, 2)); System.out.println("test_isPrime()"); } }
Output :
@BeforeAll @BeforeEach @BeforeEach test_Multiply() @BeforeEach
1.2. assertTimeout() or assertTimeoutPreemptively()
1.2.1. assertTimeout()
– Test method will be failed if not executed within time, but assertTimeout()
completes its execution process.
1.2.2. assertTimeoutPreemptively()
– Test method will be failed if not executed within time, assertTimeoutPreemptively()
will be aborted execution process. This is useful to test transaction management tests, to ensure that abort the transaction operation if it exceeds the time.
@Test void test_Add() throws Exception { assertTimeout(Duration.ofSeconds(5), () -> { TimeUnit.SECONDS.sleep(4); assertEquals(5, MathUtil.add(3, 2)); System.out.println("test_Add()"); }); } @Test void test_Multiply() throws Exception { // fails but completes the process, see output and run duration assertTimeout(Duration.ofSeconds(5), () -> { TimeUnit.SECONDS.sleep(6); assertEquals(5, MathUtil.add(3, 2)); System.out.println("test_Multiply()"); }); } @Test void test_assertTimeoutPreemptively() throws Exception { // fails but abort the process, see output and run duration assertTimeoutPreemptively(Duration.ofSeconds(5), () -> { TimeUnit.SECONDS.sleep(6); assertEquals(5, MathUtil.add(3, 2)); System.out.println("test_assertTimeoutPreemptively()"); }); }
Output :
test_Add() test_Multiply()
Execution Time and status of each Test :
- test_Add() – Passed – 4.035 s
- test_Multiply() – Failed – 10.011 s
- test_assertTimeoutPreemptively() – Failed – 5.016 s
1.2.3. Declaring @Timeout
on a @TestFactory
method checks that the factory method returns within the specified duration but does not verify the execution time of each individual DynamicTest
generated by the factory. We can use assertTimeout()
or assertTimeoutPreemptively()
for that purpose.
Junit 5 Timeout for Dynamic tests example :
@TestFactory @Timeout(5) Collection<DynamicTest> test_dynamicTestsTimeouts() { return Arrays.asList( dynamicTest("1st dynamic test", () -> { TimeUnit.SECONDS.sleep(3); assertEquals(5, MathUtil.add(3, 2)); System.out.println("Dynamic Test 1"); }), dynamicTest("2nd dynamic test", () -> { TimeUnit.SECONDS.sleep(2); assertEquals(6, MathUtil.multiple(3, 2)); System.out.println("Dynamic Test 2"); })); } @TestFactory Collection<DynamicTest> test_dynamicTests_AssertTimeouts() { return Arrays.asList( dynamicTest("3rd dynamic test", () -> { assertTimeout(Duration.ofSeconds(5), () -> { TimeUnit.SECONDS.sleep(10); assertEquals(5, MathUtil.add(3, 2)); System.out.println("Dynamic Test 3"); }); }), dynamicTest("4th dynamic test", () -> { assertTimeoutPreemptively(Duration.ofSeconds(5), () -> { TimeUnit.SECONDS.sleep(10); assertEquals(5, MathUtil.add(3, 2)); System.out.println("Dynamic Test 4"); }); })); }
Output :
Dynamic Test 1 Dynamic Test 2 Dynamic Test 3
Execution Time and status of each Test :
- 1st dynamic test – Passed – 3.006 s
- 2nd dynamic test – Passed – 2.004 s
- 3rd dynamic test – Failed – 10.004 s
- 4th dynamic test – Failed – 5.003 s
1.3. Junit 5 global timeouts
Configuration parameters can be used to specify global tiemouts. For example to specify timeouts for all the @BeforeEach
methods in entire test application set the value for configuration property junit.jupiter.execution.timeout.beforeeach.method.default
Following can be used to specify global timeouts. Values can be set in junit-platform.properties
file under test/resources
folder or can be set in maven sure-fire plugin configurationParameters
properties or can set as System property value can be provide from maven command.
junit.jupiter.execution.timeout.default
– Default timeout for all testable and lifecycle methods.junit.jupiter.execution.timeout.testable.method.default
– Default timeout for all testable methods.junit.jupiter.execution.timeout.test.method.default
– Default timeout for@Test
methods.junit.jupiter.execution.timeout.testtemplate.method.default
– Default timeout for@TestTemplate
methods.junit.jupiter.execution.timeout.testfactory.method.default
– Default timeout for@TestFactory
methods.junit.jupiter.execution.timeout.lifecycle.method.default
– Default timeout for all lifecycle methods.junit.jupiter.execution.timeout.beforeall.method.default
– Default timeout for@BeforeAll
methods.junit.jupiter.execution.timeout.beforeeach.method.default
– Default timeout for@BeforeEach
methods.junit.jupiter.execution.timeout.afterall.method.default
– Default timeout for@AfterAll
methods.junit.jupiter.execution.timeout.aftereach.method.default
– Default timeout for@AfterEach
methods.
Example junit-platform.properties
:
junit.jupiter.execution.timeout.beforeeach.method.default=200 ms junit.jupiter.execution.timeout.test.method.default=10 s
2. Disable @Timeout globally
When we are debugging test cases timeout may influence the results of test execution. For example assume that, when writing tests we set the timeout value 10 seconds to complete transactions related tests, but later it increased to 20 seconds. All the tests may impact and mark the test as failed although all assertions were met.
JUnit Jupiter supports the junit.jupiter.execution.timeout.mode
configuration parameter to configure when timeouts are applied. There are three modes.
enabled
disabled
disabled_on_debug
Example junit-platform.properties
:
junit.jupiter.execution.timeout.mode=disabled
Conclusion
In this article we have seen what are the Junit 5 timemouts, how to use @Timeout
in writing test cases, how to use global timeouts and how to disable timeouts globally with several examples. Declarative timeouts are an experimental feature, but this is very useful feature to handle timeouts globally.