Integration testing is an important part of any software development process. It is a phase of software testing where different components of a system are tested together as a group. This helps ensure that the different components work together as expected. With the rise of micro-services and containerization, Integration testing has become even more crucial. In this article, we will explore how to use TestContainers and docker-compose to perform integration testing on a Spring Boot application.
TestContainers is a Java library that allows you to run docker containers as part of your integration tests. It provides a simple API to start and stop these containers. This makes it easy to test how your application interacts with external services, such as databases or message queues. You can run lightweight, disposable instances of databases, browsers, and other services in ephemeral Docker containers during your tests. This improves the reliability, speed, and ease of writing end-to-end tests for applications that use external dependencies.
With TestContainers, you can spin up a real database or another service without having to install and configure it on your local machine, which can reduce the complexity of your test setup and make your tests more robust and consistent.
Docker-compose is a tool for defining and running multi-container applications. It uses a YAML file to define the services that make up an application, and their configurations. With TestContainers, you can use a docker-compose file to define the services that your tests depend on.
Why TestContainers?
Before I started using TestContainers, I used to run my testing on H2 database and mock out external dependencies. Needless to say, this was not the most effective form of testing. It worked most of the time, but I would often run into issues in production that were peculiar to Mysql or MongoDB depending on my production database of choice. Also, if I was testing within a MicroService environment, I could discover issues when integration between services needed to be utilized; issues that didn’t arise in testing.
TestContainers provides a way to instantiate docker containers of external dependencies in your integration testing. So instead of using H2 database for my tests, I can start a MySQL docker image and point my tests to run against MySQL.
Setting up
To get started, you will need to add TestContainers as a dependency to your project. You can do this by adding the following to your pom.xml file:
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>mysql</artifactId>
<version>1.17.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<version>1.17.2</version>
<scope>test</scope>
</dependency>Once you’ve done this, you can create a test that uses TestContainers to start a container. For example, the following test starts a MySQL container, and then uses the container’s IP address and port to connect to the database:
@Test
void testWithMySQL() {
// Start the MySQL container
var mysqlContainer = new MySQLContainer<>("mysql:8.0")
.withDatabaseName("mydb")
.withUsername("username")
.withPassword("password");
mysqlContainer.start();
// Connect to the database using the container's IP address and port
var jdbcUrl = "jdbc:mysql://" + mysql.getContainerIpAddress() + ":" + mysql.getMappedPort(3306) + "/mydb";
try (Connection connection = DriverManager.getConnection(jdbcUrl, "username", "password")) {
// Run your tests here
}
// Stop the container
mysql.stop();
}
You can also use a docker-compose file to define the services that your tests depend on. This makes it easy to test how your application interacts with multiple services. For example, the following docker-compose file defines a MySQL and a RabbitMQ service:
version: '3'
services:
mysql:
image: mysql:8.0
environment:
MYSQL_DATABASE: mydb
MYSQL_USER: username
MYSQL_PASSWORD: password
MYSQL_ROOT_PASSWORD: password
ports:
- "3306:3306"
healthcheck:
test: "mysql $$MYSQL_DATABASE -uroot -p$$MYSQL_ROOT_PASSWORD -e 'SELECT 1;'"
interval: 10s
timeout: 300s
retries: 10
rabbitmq:
image: rabbitmq:3-management
ports:
- "5672:5672"
- "15672:15672"
You can then use the DockerComposeContainer to start and stop the services defined in the docker-compose file. For example, the following test starts the MySQL and RabbitMQ services, and then uses the container’s IP address and port to connect to them:
@Test
void testWithDockerCompose() {
// Start the services defined in the docker-compose file
var composeContainer = new DockerComposeContainer(new File("src/test/resources/docker-compose.yml"))
.withExposedService("mysql", 3306)
.withExposedService("rabbitmq", 5672);
composeContainer.start();
// Connect to the MySQL service using the container's IP address and port
var jdbcUrl = "jdbc:mysql://" + composeContainer.getServiceHost("mysql", 3306) + ":" + composeContainer.getServicePort("mysql", 3306) + "/mydb";
try (Connection connection = DriverManager.getConnection(jdbcUrl, "username", "password")) {
// Run your tests here
}
// Connect to the RabbitMQ service using the container's IP address and port
var factory = new ConnectionFactory();
factory.setHost(composeContainer.getServiceHost("rabbitmq", 5672));
factory.setPort(composeContainer.getServicePort("rabbitmq", 5672));
try (Connection connection = factory.newConnection()) {
// Run your tests here
}
// Stop the services
composeContainer.stop();
}
The above examples start and stop the containers within the test method. But this might not be ideal if you want to run a suite of tests. There are different strategies you can employ to utilize TestContainers across your entire test suite:
- Declare Containers as beans.
- Declare Containers in a Test superclass and annotate with @Container.
- Use the Singleton pattern to define containers in a Test superclass and start manually.
Let’s look at each of them.
Declaring Containers as beans
Not my favorite approach, but this works fine. Here I set up MySQLContainer as a spring bean and Autowire in my Datasource bean. So the data source available in my test context is pointing to the MySQLContainer.
@TestConfiguration
public class TestDataSourceConfig {
/**
* create mysql container as a bean. start the container before returning it.
*/
@Bean
public MySQLContainer mySQLContainer() {
var container = new MySQLContainer<>("mysql:8.0")
.withDatabaseName("mydb")
.withUsername("username")
.withPassword("password")
.withUrlParam("createDatabaseIfNotExist", "true")
.withUrlParam("serverTimezone", "UTC")
.withReuse(true)
.withLogConsumer(
new Slf4jLogConsumer(
LoggerFactory.getLogger(getClass())
)
);
container.start();
return container;
}
/**
* autowire mysql container in datasource config and use container properties
* to initialise the datasource.
*/
@Bean
@Primary
public DataSource dataSource(MySQLContainer mySQLContainer) {
var dataSource = new HikariDataSource();
dataSource.setJdbcUrl(mySQLContainer.getJdbcUrl());
dataSource.setUsername(mySQLContainer.getUsername());
dataSource.setPassword(mySQLContainer.getPassword());
dataSource.setDriverClassName(mySQLContainer.getDriverClassName());
return dataSource;
}
}
The datasource configuration above can be included in a base integration class. Something like this:
@ExtendWith({SpringExtension.class})
@SpringBootTest(
classes = {
TestDataSourceConfig.class,
},
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT
)
public abstract class BaseIntegrationTest {
}
So long as your Test classes extend the BaseIntegrationTest, they should reuse the same context which should have the MySQLContainer initialized with the Datasource bean.
Use @Container annotation with @TestContainers configuration
If you would like TestContainers to manage your containers as part of your test life cycles, then it’s best to use this approach. You will have to annotate the Integration test class with @TestContainers which will look for fields declared with @Container annotation and automatically start and stop them as needed.
Using the @DynamicPropertySource annotation which was introduced in Spring 5.2.5, we can dynamically set properties in our spring environment. In the example below, I am able to set the ‘spring.datasource.url’ property dynamically after my docker-compose containers have been initialized.
@Slf4j
@ExtendWith({SpringExtension.class})
@SpringBootTest
@Testcontainers
public abstract class BaseIntegrationTest {
@Container
private static DockerComposeContainer dockerComposeContainer = new DockerComposeContainer<>(
new File("src/test/resources/docker-compose.yml")
).withExposedService(
"mysqldb_1", 3306, Wait.forHealthcheck()
);
@DynamicPropertySource
protected static void setProperties(
DynamicPropertyRegistry registry
) {
setDataSourceProperties(registry);
}
private static void setDataSourceProperties(DynamicPropertyRegistry registry) {
var mysqlDbHost = dockerComposeContainer.getServiceHost("mysqldb_1", 3306);
var mysqlDbPort = dockerComposeContainer.getServicePort("mysqldb_1", 3306);
log.info("DB Connection Properties: {}, {}", mysqlDbHost, mysqlDbPort);
registry.add(
"spring.datasource.url",
() -> String.format("jdbc:mysql://%s:%d/mydb?createDatabaseIfNotExist=true", mysqlDbHost, mysqlDbPort)
);
}
}
Use Singleton pattern to define containers in a Test superclass and start manually.
We can also declare our container in a static field and start it manually in a static code block. This is useful if you want your container started only once for several classes extending a base class.
@Slf4j
@ExtendWith({SpringExtension.class})
@SpringBootTest
public abstract class BaseIntegrationTest {
private static DockerComposeContainer dockerComposeContainer = new DockerComposeContainer<>(
new File("src/test/resources/docker-compose.yml")
).withExposedService(
"mysqldb_1", 3306, Wait.forHealthcheck()
);
static {
dockerComposeContainer.start();
}
@DynamicPropertySource
protected static void setProperties(
DynamicPropertyRegistry registry
) {
setDataSourceProperties(registry);
}
private static void setDataSourceProperties(DynamicPropertyRegistry registry) {
var mysqlDbHost = dockerComposeContainer.getServiceHost("mysqldb_1", 33080);
var mysqlDbPort = dockerComposeContainer.getServicePort("mysqldb_1", 33080);
log.info("DB Connection Properties: {}, {}", mysqlDbHost, mysqlDbPort);
registry.add(
"spring.datasource.url",
() -> String.format("jdbc:mysql://%s:%d/mydb?createDatabaseIfNotExist=true", mysqlDbHost, mysqlDbPort)
);
}
}
Running your tests with TestContainers and docker-compose can allow you to test with entire services interacting with each other in an isolated environment. You can spin up services, message queues, redis instances, in order to mimic a production environment and do a full integration test end-to-end.
Conclusion
Testing using TestContainers can be more expensive to operate, as opposed to mocking service interactions. You may notice increased startup times for your tests. In the long run, however, it can be more rewarding if done correctly. TestContainers offers better integration testing as you can test features end-to-end across all connected components.
TestContainers and Docker Compose are powerful tools that can greatly improve the reliability and speed of writing end-to-end tests for applications that use external dependencies. TestContainers allows you to easily spin up real instances of databases, browsers, and other services in ephemeral Docker containers during your tests, making your test setup more robust and consistent. Meanwhile, Docker Compose allows you to define and run multi-container applications, which can be useful for testing complex applications that depend on multiple services. Together, Testcontainers and Docker Compose can provide a powerful and efficient solution for testing applications that use external dependencies, making it a valuable tool for any developer or tester.