Integration tests with Testcontainers

We explore how to write integration tests using Testcontainers.org library in Java-based backend applications.

Senior Software Engineer

Posted on Feb 25, 2021

Tags:

In this article, I will show how teams at Zalando Marketing Services are using integration tests in Java-based backend applications. We will follow the idea of integration tests: the main concept and the attributes of a good integration test. Then, we will discuss an example based on the TestContainers library used in the Spring environment.

Integration tests

There are many definitions of integration testing. For example, the definition found on Wikipedia is: Integration testing is the phase in software testing in which individual software modules are combined and tested as a group.

For this article, we define integration tests as tests of communication between our code and external components, e.g. database, one of the AWS services (like S3, Kinesis, DynamoDB, SQS, and others) or an external system with which we are communicating over HTTP.

The purpose of integration tests is to assess how our code will behave when communicating with external services. Not only in happy path scenarios, but especially in corner cases, e.g. external service will respond with an unexpected HTTP code, the HTTP response will come after a defined timeout, AWS S3 responses with internal errors.

Amount of integration tests

While implementing tests, we need to remember to maintain the proper balance between different test types. Integration tests cannot be the core of the testing codebase.

A pyramid of testing shows us the proportions of different types of tests. For backend applications, the foundations are unit tests and component tests. Integration tests are a complement of unit tests and other test types like component, system, and manual.

Pyramid of testing

System tests and manual tests should ideally be the rarest type of tests. From our experience, we estimate the number of integration tests to be around 25% of unit tests, but it varies from application to application.

Integration tests with Testcontainers library

Let's see how to organize an integration test with the Testcontainers library, and how to manage a startup/teardown of Docker containers. Testcontainers.org is a JVM library that allows users to run and manage Docker images and control them from Java code. Zalando uses it mainly for integration tests. To implement an integration test, you need to run your application similarly to a unit test (method annotated by @Test).

The integration test additionally runs external components as real Docker containers. External components can be one of:

database storage - for example, run real PostgreSQL as a Docker image,
mocked HTTP server - you can mimic the behavior of other HTTP services by using Docker images from MockServer or WireMock,
Redis - run real Redis as a Docker image,
streams or queues (like RabbitMQ and others),
AWS components like S3, Kinesis, DynamoDB, and others, which you can emulate with Localstack
other application that can be run as a Docker image.

It is very easy to run Docker images from Java code. Every Docker image can be run with GenericContainer. For the most popular Docker images, there are prepared wrapper classes for convenient usage.

To make sure that every Docker image will be stopped after usage and resources are released, the library uses JVM ShutdownHooks and a special Docker image Ryuk. ShutdownHooks stops images when tests are finished. In case the Java process is no longer available, the Ryuk container stops all Docker images. It is worth mentioning that it is possible to disable Ryuk containers.

Your service communicates with external components run as Docker images.

Maven configuration

To use Testcontainers, add a maven dependency with a current library version.

<dependency>
  <groupId>org.testcontainers</groupId>
  <artifactId>testcontainers</artifactId>
  <version>${testcontainers.version}</version>
  <scope>test</scope>
</dependency>

It's important to have control over test execution. Unit tests should be executed before integration tests. It is a consequence of the pyramid of testing and helps to ensure that feedback loops are short. In some cases, you may want to skip integration tests, for example when your local machine is slow and you want to run it only on CI/CD.

To run the integrations tests after your unit tests, simply add maven-failsafe-plugin to your project. Failsafe and Surefire plugins work in different build phases. By default, the Maven Surefire plugin executes unit tests during the test phase. It includes all classes whose name ends with Test / Tests or TestCase. The Failsafe plugin runs integration tests in the integration-test phase. To separate execution, we configure Failsafe plugin to run classes with postfix IntegrationTest. We also create a special profile, here: with-integration-tests to control if we want to run integration-tests or not.

<profiles>
 <profile>
   <id>with-integration-tests</id>
   <build>
     <pluginManagement>
       <plugins>
         <plugin>
           <groupId>org.apache.maven.plugins</groupId>
           <artifactId>maven-failsafe-plugin</artifactId>
           <executions>
             <execution>
               <goals>
                 <goal>integration-test</goal>
                 <goal>verify</goal>
               </goals>
             </execution>
           </executions>
           <configuration>
             <includes>
               <include>**/*IntegrationTest.java</include>
             </includes>
           </configuration>
         </plugin>
       </plugins>
     </pluginManagement>
   </build>
 </profile>

An invocation of maven command would look like:

mvn clean verify -P with-integration-tests

Basic integration test with TestContainers

Let’s set up a basic integration test with JUnit 5 and Spring Boot.

An integration test class example can look like the example below. The test class inherits from AbstractIntegrationTest. The test method creates an entity in the database run as a Docker image. Later, we read the entity from the database and control if the entity has been written correctly.

class AccountRepositoryIntegrationTest extends AbstractIntegrationTest {

    @Autowired
    private AccountRepository dao;

    @Test
    void shouldCreateAccount() {
        // given
        Account account = createAccount();

        // when
        underTest.save(account);

        // then
        Optional<Account> actualOptional = dao.findById(account.getId());
        Account expected = createAccount();
        assertThat(actualOptional).isPresent();
        assertThat(actualOptional.get()).isEqualTo(expected);
    }
}

The test class below is an abstract class that will be inherited by all integration tests. It contains static references to Docker containers - singleton container. In the static block, we start all images. We do not need to stop them, it will be done automatically. In the example below, the PostgreSQLContainer is going to listen on a random port. To facilitate adding properties with dynamic values, we used the @DynamicPropertySource annotation that was introduced in Spring Framework 5.2.5 (it has a more compact syntax than ApplicationContextInitializer).

@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@ActiveProfiles("test")
public abstract class AbstractIntegrationTest {

    public static PostgreSQLContainer postgreSQL =
      new PostgreSQLContainer("postgres:13.1")
            .withUsername("testUsername")
            .withPassword("testPassword")
            .withDatabaseName("testDatabase");

    static {
        postgreSQL.start();
    }

    @DynamicPropertySource
    static void postgresqlProperties(DynamicPropertyRegistry registry) {
        registry.add("db_url", postgreSQL::getJdbcUrl);
        registry.add("db_username", postgreSQL::getUsername);
        registry.add("db_password", postgreSQL::getPassword);
    }
}

@TestContainers annotation

There are also different ways of running your containers. You can use the annotations set prepared in the Junit-Jupiter maven module:

<dependency>
 <groupId>org.testcontainers</groupId>
 <artifactId>junit-jupiter</artifactId>
 <version>${testcontainers.version}</version>
 <scope>test</scope>
</dependency>

A test class annotated with the @Testcontainers annotation runs all containers annotated with the @Container annotation. Additionally, when the container is static, it shares containers between test methods. You can control the startup order of containers by using dependsOn method of GenericContainer. The main limitation is, that containers cannot be reused between test classes. Moreover, this extension has only been tested with sequential test execution. Using it with parallel test execution is unsupported and may have unintended side effects. The test class would look like the example below.

@Testcontainers
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@ActiveProfiles("test")
public class ApplicationIntegrationTest {

    @Container
    public static PostgreSQLContainer postgreSQL =
      new PostgreSQLContainer("postgres:13.1")
            .withUsername("testUsername")
            .withPassword("testPassword")
            .withDatabaseName("testDatabase");

  @DynamicPropertySource
  static void postgresqlProperties(DynamicPropertyRegistry registry) {
    registry.add("spring.datasource.url", postgreSQL::getJdbcUrl);
    registry.add("spring.datasource.password", postgreSQL::getPassword);
    registry.add("spring.datasource.username", postgreSQL::getUsername);
  }

  @Test
  public void contextLoads() {
  }

}

Lifecycle of integration test

All tests (including integration tests) should follow principles defined as FIRST. The acronym FIRST was defined in the book Clean Code written by Robert C. Martin.

[F]ast - A test should not take more than a second to finish the execution.
[I]solated - No order-of-run dependency.
[R]epeatable - A test method should NOT depend on any data in the environment/instance in which it is running.
[S]elf-Validating - No manual inspection required to check whether the test has passed or failed.
[T]horough - Should cover every use case scenario and NOT just aim for 100% coverage.

Running a Docker image for every test method can take an enormous amount of time. To increase performance we need to make a real-life compromise. We can run a Docker image per class or even run once for all integration test executions. The second approach has been presented in the code. If we decide to share Docker images between tests, we need to be ready for it. There are many ways to achieve it, e.g.:

Tests should operate on unique IDs, names, etc. That way, we can avoid collisions of database constraints. In this case, you don’t need to clean up after the test execution. Some problems can occur, for example when you count elements in the database table. You can count elements created by different tests.
Tests should clean up the state after execution. This approach consumes much more development time and is error-prone.

If we would like to run tests concurrently, it would require even more discipline from developers.

Advantages of using the TestContainers library

You run tests against real components, for example, the PostgreSQL database instead of the H2 database, which doesn’t support the Postgres-specific functionality (e.g. partitioning or JSON operations).
You can mock AWS services with Localstack or Docker images provided by AWS. It will simplify administrative actions, cut costs and make your build offline.
You can run your tests offline - no Internet connection is needed. It is an advantage for people who are traveling or if you have a slow Internet connection (when you have already run them once and there is no version change in the container).
You can test corner cases in HTTP communication like:
- programmatically simulate timeout from external services (e.g. by configuring MockServer to respond with a delay that is bigger than the timeout set in your HTTP client),
- simulate HTTP codes that are not explicitly supported by our application.
Implementation and tests can be written by developers and exposed in the same pull request by backend developers.
Even one integration test can verify if your application context starts properly and your database migration scripts (e.g. Flyway) are executing correctly.

Disadvantages of using the TestContainers library

We bring another dependency to our system that you need to maintain.
You need to run containers at least once - it consumes time and resources. For example, PostgreSQL as a Docker image needs around 4 seconds to start on my machine, whereas the H2 in-memory database needs only 0.4 seconds. From my experience, Localstack which emulates AWS components, can start much longer, even 20 seconds on my machine.
A continuous integration (e.g. Jenkins) machine needs to be bigger (build uses more RAM and CPU).
Your local computer should be pretty powerful. If you run many Docker images, it can consume a lot of resources.
Sometimes, integration tests with TestContainers are still not sufficient. For example, while testing REST responses with a mockserver container you can miss changes of real API. Inside the integration test, you may not reflect it, and your code still can crash on production. To minimize the risk, you may consider leveraging Contract Testing via Spring Cloud Contract.