Introduction

Testcontainers for Go enables developers to run easily tests against containerized dependencies. In our previous articles, you can find an introduction of Integration tests with Testcontainers and explore how to write Functional tests with Testcontainers (in Java).

This blog post will deep dive into how to use modules and a common issue for Testcontainers with Golang.

What we use it for?

Services often use external dependencies like datastore or queues. It is possible to mock these dependencies but if you want to run for example integration test, it is better to verify against the real dependency (or close enough).

Starting a container with the image of the dependency is a convenient way to verify that the application works as expected. With Testcontainers, starting the container is done programmatically so that you can define it as part of your tests. The machine running the tests (developer, CI/CD) requires to have a container runtime interface (e.g. Docker, Podman...)

Basic implementation

Testcontainers for Go is very easy to use, the quick start example is:

ctx := context.TODO()
req := testcontainers.ContainerRequest{
    Image:        "redis:latest",
    ExposedPorts: []string{"6379/tcp"},
    WaitingFor:   wait.ForLog("Ready to accept connections"),
}
redisC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
    ContainerRequest: req,
    Started:          true,
})
if err != nil {
    panic(err)
}
defer func() {
    if err := redisC.Terminate(ctx); err != nil {
        panic(err)
    }
}()

If we dive into the code above, we notice that:

1. testcontainers.ContainerRequest initialises a struct with container image, exposed port and waiting strategy parameters
2. testcontainers.GenericContainer starts the container returning the container and error structs
3. redisC.Terminate terminates the container with defer once the test is done

Implementing our own internal library

From the example in the previous section, there is some minor inconvenience:

1. wait.ForLog("Ready to accept connections") uses logs to wait for start of the container which can break easily
2. ExposedPorts: []string{"6379/tcp"} requires knowledge of the exposed port for Redis

There might also be some additional environment variables and other parameters useful to run a Redis container which requires deeper knowledge. As such, we decided to create an internal library which would initialise container with the default parameters required to ease test implementation. To remain flexible, we used the Functional Options Pattern so that consumer can still customize depending on the needs.

Example of implementation for Redis:

func defaultPreset() []container.Option {
    return []container.Option{
        container.WithPort("6379/tcp"),
        container.WithGetURL(func(port nat.Port) string {
            return "localhost:" + port.Port()
        }),
        container.WithImage("redis"),
        container.WithWaitingStrategy(func(c *container.Container) wait.Strategy {
            return wait.ForAll(
                wait.NewHostPortStrategy(c.Port),
                wait.ForLog("Ready to accept connections"))
        }),
    }
}

// New - create a new container able to run redis
func New(options ...container.Option) (*container.Container, error) {
    c := container.Container{}
    options = append(defaultPreset(), options...)
    for _, o := range options {
        o(&c)
    }

    return &c, nil
}

// Start - start a Redis container and return a container.CreatedContainer
func Start(ctx context.Context, options ...container.Option) (container.CreatedContainer, error) {
    p, err := New(options...)
    if err != nil {
        return container.CreatedContainer{}, err
    }
    return p.Start(ctx)
}

Usage of the library for Redis:

ctx := context.TODO()
cc, err := redis.Start(ctx, container.WithVersion("latest"))
if err != nil {
    panic(err)
}
defer func() {
    if err := cc.Stop(ctx, nil); err != nil {
        panic(err)
    }
}()

With this internal library, developers could easily add tests for Redis without the need to figure out the waiting strategy, exposed port, etc. In case of incompatibility, the internal library could be updated to centrally fix the issue.

Common issue - Garbage collector (Ryuk / Reaper)

Testcontainers covers the extra mile of ensuring that container is removed once test is done using a Garbage Collector which is an additional container started as a "sidecar". This container is responsible for stopping the container being tested even if your test crash (which would prevent defer to run).

When using Docker, it works without problem, but with other container runtime interfaces (like Podman) often you will get this kind of error: Error response from daemon: container create: statfs /var/run/docker.sock: permission denied: creating reaper failed: failed to create container.

One way to "fix this" is to deactivate it with the environment variable TESTCONTAINERS_RYUK_DISABLED=true.

Another way is to set the Podman machine rootful and add:

export TESTCONTAINERS_RYUK_CONTAINER_PRIVILEGED=true; # needed to run Reaper (alternative disable it TESTCONTAINERS_RYUK_DISABLED=true)
export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock; # needed to apply the bind with statfs

In our internal library we took the approach of disabling it by default as developers had issues running it locally.

Moving to modules

Once our internal library was stable enough, we decided that it was time to give back to the community by contributing to Testcontainers. But surprise... modules has just been introduced in Testcontainers. Module is doing exactly what our internal library was for, we therefore migrated all our services to modules and discontinued the internal library. From the migration, we learned that it was possible to use the standard library out of the box now that modules have been introduced, which reduces the maintenance cost of our services. The main challenge was to fine-tune developer environment variables to run on the developer machine (make Garbage Collector work) using Makefile.

Adapted example from testcontainers documentation:

ctx := context.TODO()
redisContainer, err := redis.RunContainer(ctx,
    testcontainers.WithImage("docker.io/redis:latest"),
)
if err != nil {
    panic(err)
}
defer func() {
    if err := redisContainer.Terminate(ctx); err != nil {
        panic(err)
    }
}()

Conclusion

Testcontainers for Golang is a great library to support testing which is even better now that modules have been introduced. Some small impediments with the Garbage collector exist, but that can be fixed easily as described in this post.

I hope with this blog, if you haven't already, that you will adopt Testcontainers, highly recommended to improve testability of your applications.

We're hiring! Do you like working in an ever evolving organization such as Zalando? Consider joining our teams as a Software Engineer!