Introduction 🚀

Hey there, fellow engineers! Let's dive into the exciting world of Spring Boot testing with JUnit. It is incredibly powerful, providing a realistic environment for testing our code. However, if we don't optimize our tests, they can be slow and negatively affect lead time to changes for our teams.

This blog post will teach you how to optimize your Spring Boot tests, making them faster, more efficient, and more reliable.

Imagine an application whose tests take 10 minutes to execute. That's a lot of time! Let's roll up our sleeves and see how we can whiz through those tests in no time! 🕒✨

Understanding Test Slicing in Spring

Test slicing in Spring allows testing specific parts of an application, focusing only on relevant components, rather than loading the entire context. It is achieved by annotations like @WebMvcTest, @DataJpaTest, or @JsonTest. These annotations are a targeted approach to limit the context loading to a specific layer or technology. For instance, @WebMvcTest primarily loads the Web layer, while @DataJpaTest initializes the Data JPA layer for more concise and efficient testing. This selective loading approach is a cornerstone in optimizing test efficiency.

There are more annotations that can be used to slice the context. See official Spring documentation on Test Slices.

Test Slicing: Using @DataJpaTest as a replacement for @SpringBootTest 🧩

Let's take a look at an example (code below). The test first deletes all the data (shipments and containers, each shipment can have multiple containers) from the target tables, and then saves a new shipment. Next, it creates a thread pool with 50 threads, where each thread calls the svc.createOrUpdateContainer method.

The test will wait until all the threads are finished, then it will check that the database has only one container.

It's all about checking concurrency issues and involves a swarm of threads, clocking in at about 16 seconds on my machine – a massive chunk of time for a single service check, right?

@ActiveProfiles("test")
@SpringBootTest
abstract class BaseIT {
    @Autowired
    private lateinit var shipmentRepo: ShipmentRepository

    @Autowired
    private lateinit var containerRepo: ContainerRepository
}

class ContainerServiceTest : BaseIT() {
    @Autowired
    private lateinit var svc: ContainerService

    @BeforeEach
    fun setup() {
        shipmentRepo.deleteAll()
        containerRepo.deleteAll()
        shipmentRepo.save(shipment)
    }
    @Test
    fun testConcurrentUpdatesForContainer() {

        val executor = Executors.newFixedThreadPool(50)
        repeat(50) {
            executor.execute {
                containerService.createOrUpdateContainer("${shipment.id}${svc.DEFAULT_CONTAINER}", Patch("NEW_LABEL"))
            }
        }
        executor.shutdown()
        while (!executor.awaitTermination(100, TimeUnit.MILLISECONDS)) {
            // busy waiting for executor to terminate
        }
        assertThat(containerRepo.find(shipment)).hasSize(1)
    }

}

The first problem we have is the class declaration:

class ContainerServiceTest : BaseIT()

The issue starts with the BaseIT class using @SpringBootTest. This causes the Spring context for the entire application to be loaded (every time we mess with context caching mechanisms, we'll get to that later!). When the application is large enough, a huge number of beans are loaded - a costly operation for tests with specific objectives.

But no, we don't want to load everything. All we need to load is the ContainerService bean and JPA repositories. We can switch to @DataJpaTest. This annotation only loads the JPA part of the application, which is what we need for this test. Let's try it out!

@DataJpaTest
class ContainerServiceTest {
    @Autowired
    private lateinit var svc: ContainerService

    @Autowired
    private lateinit var shipmentRepo: ShipmentRepository

    @Autowired
    private lateinit var containerRepo: ContainerRepository
}

Upon execution, an exception is thrown:

org.springframework.beans.factory.BeanCreationException: Failed to replace DataSource with an embedded database for tests. If you want an embedded database please put a supported one on the classpath or tune the replace attribute of @AutoConfigureTestDatabase.

@DataJpaTest has an annotation @AutoConfigureTestDatabase, which by default, sets up an H2 in-memory database for the tests, and configures DataSource to use it. However, in this case, the H2 dependency is not found in the classpath.

And actually, we don't want to use H2 for our tests, so we can tell @AutoConfigureTestDatabase not to replace our configured database with an H2. Plus, we have to configure and load our own database, which is performed here by importing a @Configuration class called EmbeddedDataSourceConfig (It simply creates a @Bean of type DataSource).

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Import(EmbeddedDataSourceConfig::class) // Import the embedded database configuration if needed.
@ActiveProfiles("test") // Use the test profile to load a different configuration for tests.
class ContainerServiceTest {
    // test code
}

Let's try to run the test again. Now, it fails with this error:

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'ContainerServiceTest': Unsatisfied dependency expressed through field 'containerService'

You already know the trick, you need to load the ContainerService bean in the Spring context!

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Import(ContainerService::class, EmbeddedDataSourceConfig::class)
@ActiveProfiles("test")
class ContainerServiceTest {
    // test code
}

Uh-oh! The Spring context loads successfully, but the test fails with the following error:

java.lang.AssertionError:
Expected size:<1> but was:<0> in:
<[]>

If you look at @DataJpaTest, you will notice that it uses the @Transactional annotation. It means that by default, deleting data from the target tables and creating a new container will only be committed at the end of the test method, thus the changes are not visible to the transactions created by the threads.

Since we would like to commit the transaction inside the main transaction (which @DataJpaTest uses), we need to use Propagation.REQUIRES_NEW:

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Import(ContainerService::class, EmbeddedDataSourceConfig::class)
@ActiveProfiles("test")
class ContainerServiceTest {
    @Autowired
    private lateinit var transactionTemplate: TransactionTemplate

    @Autowired
    private lateinit var svc: ContainerService

    @Autowired
    private lateinit var shipmentRepo: ShipmentRepository

    @Autowired
    private lateinit var containerRepo: ContainerRepository

    @BeforeEach
    fun setup() {
        transactionTemplate.propagationBehavior = TransactionTemplate.PROPAGATION_REQUIRES_NEW
        transactionTemplate.execute {
            shipmentRepo.deleteAll()
            containerRepo.deleteAll()
            shipmentRepo.save(shipment)
        }
    }

}

🎉 The test passes, completing in just 8 seconds (load context + run) - twice as fast as before!

Test Slicing: @JsonTest Precision in Validating JSON Serialization/Deserialization 💡

Consider this test snippet:

    public class EventDeserializationIT extends BaseIT {

    private static final String RESOURCE_PATH = "event-example.json";

    @Autowired
    private ObjectMapper objectMapper;

    private Event dto;

    @Test
    public void testDeserialization() throws Exception {
        String json = Resources.toString(Resources.getResource(RESOURCE_PATH), UTF_8);
        dto = objectMapper.reader().forType(Event.class).readValue(json);

        assertThat(dto.getData().getNewTour().getFromLocation()).isNotNull();
        assertThat(dto.getData().getNewTour().getToLocation()).isNotNull();
    }
}

The objective of this test is to ensure proper deserialization. We can use @JsonTest annotation to import the beans that we need in the test. We only need object mapper, no need to extend any other classes! Using this annotation will only apply the configuration relevant to JSON tests (i.e. @JsonComponent, Jackson Module).

@JsonTest
public class EventDeserializationTest {

    @Autowired
    private ObjectMapper objectMapper;

    // Test implementation
}

Test Slicing: @WebMvcTest for REST APIs 🌐

Using @WebMvcTest, we can test REST APIs without firing up the server (e.g., the embedded Tomcat), or loading the whole application context. It’s all about targeting specific controllers. Fast and efficient, just like that!

@WebMvcTest(ShipmentServiceController.class)
public class ShipmentServiceControllerTests {

    @Autowired
    private MockMvc mvc;

    @MockBean
    private ShipmentService service;

    @Test
    public void getShipmentShouldReturnShipmentDetails() {
        given(this.service.schedule(any())).willReturn(new LocalDate());
        this.mvc.perform(
                get("/shipments/12345")
                        .accept(MediaType.APPLICATION_JSON)
                        .andExpect(status().isOk())
                        .andExpect(jsonPath("$.number").value("12345"))
                // ...
        );
    }
}

Taming Mock/Spy Beans and Context Caching Dilemmas 🔍

Let's delve into the intricacies of the Spring Test context caching mechanism!

When your tests involve Spring Test features (e.g., @SpringBootTest, @WebMvcTest, @DataJpaTest), they require a running Spring Context. Starting a Spring Context for your test requires a considerable amount of time, especially if the entire context is populated using @SpringBootTest, resulting in increased test execution overhead and longer build times if each test starts its own context.

Fortunately, Spring Test provides a mechanism to cache a started application context and reuse it for subsequent tests with similar context requirements.

The cache is like a map, with a certain capacity. The map key is computed from a few parameters, including the beans loaded into the context.

The cache key consists of:

• locations (from @ContextConfiguration)
• classes (from @ContextConfiguration)
• contextInitializerClasses (from @ContextConfiguration)
• contextCustomizers (from ContextCustomizerFactory) – this includes @DynamicPropertySource methods as well as various features from Spring Boot’s testing support such as @MockBean and @SpyBean.
• contextLoader (from @ContextConfiguration)
• parent (from @ContextHierarchy)
• activeProfiles (from @ActiveProfiles)
• propertySourceLocations (from @TestPropertySource)
• propertySourceProperties (from @TestPropertySource)
• resourceBasePath (from @WebAppConfiguration)

For example, if TestClassA specifies {"app-config.xml", "test-config.xml"} for the locations (or value) attribute of @ContextConfiguration, the TestContext framework loads the corresponding ApplicationContext and stores it in a static context cache under a key that is based solely on those locations. So, if TestClassB also defines {"app-config.xml", "test-config.xml"} for its locations (either explicitly or implicitly through inheritance) and does not define different attributes for any of the other attributes listed above, then the same ApplicationContext is shared by both test classes. This means that the setup cost for loading an application context is incurred only once (per test suite), and subsequent test execution is much faster.

If you use different attributes per different tests, for example different (ContextConfiguration, TestPropertySource, @MockBean or @SpyBean) in your test, the caching key changes. And for each new context (that does not exist in the cache), the context must be loaded from scratch.

And if there are many different contexts, the old keys from the cache are removed, thus the next running tests that could potentially use those cached contexts need to reload them. This addition results in extra test time.

One efficiency optimization method is consolidating mock beans in a parent class. This ensures that the context remains unchanged, enhancing efficiency and avoiding context reloading multiple times.

Example before and after:

@SpringBootTest
public class TestClass1 {
    @MockBean
    private DependencyA dependencyA;
    // Test implementation
}

@SpringBootTest
public class TestClass2 {
    @MockBean
    private DependencyB dependencyB;
    // Test implementation
}

@SpringBootTest
public class TestClass3 {
    @MockBean
    private DependencyC dependencyC;
    // Test implementation
}

If we tried to run the above example, the context will be reloaded 3 times, which is not efficient at all. Let's try to optimize it.

@SpringBootTest
public abstract class BaseTestClass {

    @MockBean
    private DependencyA dependencyA;

    @MockBean
    private DependencyB dependencyB;

    @MockBean
    private DependencyC dependencyC;
}

// Extend the BaseTestClass for each test class
public class TestClass1 extends BaseTestClass {

    @Test
    public void testSomething1() {
        // Test implementation
    }
}

public class TestClass2 extends BaseTestClass {

    @Test
    public void testSomething2() {
        // Test implementation
    }
}

public class TestClass3 extends BaseTestClass {

    @Test
    public void testSomething3() {
        // Test implementation
    }
}

Now, the context will be reloaded only once, which is more efficient!

Or even better: You can avoid class inheritance by using @Import annotation to import configuration classes that contain the mock beans.

@TestConfiguration
class Config {
    @MockBean
    private DependencyA dependencyA;

    @MockBean
    private DependencyB dependencyB;

    @MockBean
    private DependencyC dependencyC;
}

@Import(Config::class)
@ActiveProfiles("test")
class TestClass1 {
    // Test code
}

Think twice before using @DirtiesContext ❗

Applying @DirtiesContext to a test class removes the application context after tests are executed. This marks the Spring context as dirty, preventing Spring Test from reusing it. It's important to carefully consider using this annotation.

Although some use it to reset IDs in the database, better alternatives exist. For instance, the @Transactional annotation can be used to roll back the transaction after the test is executed.

Parallel Execution of Tests 🏎️

By default, JUnit Jupiter tests run sequentially in a single thread. However, enabling tests to run in parallel, for faster execution, is an opt-in feature introduced in JUnit 5.3. 🚀

To initiate parallel test execution, follow these steps:

1. Create a junit-platform.properties file in test/resources.

2. Add the following configuration to the file: junit.jupiter.execution.parallel.enabled = true

3. Add the following to every class you want to run parallel. @Execution(CONCURRENT)

Keep in mind that certain tests might not be compatible with parallel execution due to their nature. For such cases, you should not add @Execution(CONCURRENT). See JUnit: writing tests – parallel execution for more explanation on the different execution modes.

Results 📊

Applying all the optimizations mentioned above made a big difference in our CI/CD pipeline. Our tests are much faster, taking only 4 minutes and 15 seconds now, compared to the previous time (10 minutes 7 seconds), which is a massive 60% improvement! 🌟

Conclusion 🎬

In this adventure of optimizing Spring Boot tests, we've harnessed a collection of strategies to bolster test efficiency and speed. Let's summarize the tactics we've implemented:

• Test Slicing: Leveraging @WebMvcTest, @DataJpaTest, and @JsonTest to focus tests on specific layers or components. You can check more about (Testing Spring Boot Applications).
• Context Caching Dilemmas: Overcoming challenges related to dirty ApplicationContext caches by optimizing the use of mock and spy beans. See Spring Test Context Caching.
• Parallel Test Execution: Enabling parallel test execution to significantly reduce test suite execution time. See JUnit 5 User Guide on Parallel Execution.

These strategies collectively transform testing into a faster, more reliable, and efficient process. Each tactic, used alone or combined, contributes significantly to optimized testing practices, empowering engineers to deliver higher-quality software with enhanced efficiency.

We're hiring! Do you like working in an ever evolving organization such as Zalando? Consider joining our teams as a Software Engineer!