Run MongoDB Atlas locally for testing

Prerequisite

Java 8+
Docker

Chart shows local and remote docker support for replicaSetNumber

replicaSetNumber	local docker host	local docker host running tests from inside a container with mapping the Docker socket	remote docker daemon	availability of an arbiter node
1	+	+	+	-
from 2 to 7 (including)	only if adding either `host.docker.internal` (your Docker version should support it) or `dockerhost` to the OS host file. See Supported features for details	+	+	+

Tip: A single node replica set is the fastest among others. That is the default mode for MongoDbReplicaSet. However, to use only it, consider the Testcontainers MongoDB module on GitHub

Getting it

Gradle:

dependencies {
    testCompile("com.github.silaev:mongodb-replica-set:${LATEST_RELEASE}")
}

Maven:

<dependencies>
    <dependency>
        <groupId>com.github.silaev</groupId>
        <artifactId>mongodb-replica-set</artifactId>
        <version>${LATEST_RELEASE}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Replace ${LATEST_RELEASE} with the Latest Version Number If you cannot find a release on Maven, please, use Jitpack

Run on Apple silicon

Use digests for linux/arm64/v8 and Docker Desktop for Apple silicon supporting host.docker.internal (should be in the OS host file). Examples: 1.

MongoDbReplicaSet.builder()
    .mongoDockerImageName("mongo@sha256:8a823923d80e819e21ee6c179eabf42460b6b7d8ac3dd5f35b59419ae5413640")
    .useHostDockerInternal(true)
    .build()`

2 ./gradlew clean build -DmongoReplicaSetProperties.mongoDockerImageName=mongo@sha256:8a823923d80e819e21ee6c179eabf42460b6b7d8ac3dd5f35b59419ae5413640 -DmongoReplicaSetProperties.useHostDockerInternal=true

MongoDB versions that MongoDbReplicaSet is constantly tested against

version	transaction support
3.6.14	-
4.0.12	+
4.2.8	+
4.4.4	+
5.0.5	+

Examples

Click to see a single node example

class ITTest {
    @Test
    void testDefaultSingleNode() {
        try (
            //create a single node mongoDbReplicaSet and auto-close it afterwards
            final MongoDbReplicaSet mongoDbReplicaSet = MongoDbReplicaSet.builder()
                .mongoDockerImageName("mongo:4.4.4")
                .build()
        ) {
            //start it
            mongoDbReplicaSet.start();
            assertThat(
                mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
                hasItem(ReplicaSetMemberState.PRIMARY)
            );
            assertNotNull(mongoDbReplicaSet.getReplicaSetUrl());
        }
    }
}

Click to see a fault tolerance example

class ITTest {
    @Test
    void testFaultTolerance() {
        try (
            //create a PSA mongoDbReplicaSet and auto-close it afterwards
            final MongoDbReplicaSet mongoDbReplicaSet = MongoDbReplicaSet.builder()
                //with the latest mongo:4.4.4 docker image
                .mongoDockerImageName("mongo:4.4.4")
                //If true then use host.docker.internal of Docker, 
                //otherwise take dockerhost of Qoomon docker-host.
                //Make sure that your OS host file includes one of them.
                //All new Docker versions support the first variant.
                .useHostDockerInternal(true)
                //with 2 working nodes
                .replicaSetNumber(2)
                //with an arbiter node
                .addArbiter(true)
                //create a proxy for each node to simulate network partitioning
                .addToxiproxy(true)
                .build()
        ) {
            //start it
            mongoDbReplicaSet.start();
            assertNotNull(mongoDbReplicaSet.getReplicaSetUrl());

            //get a primary node
            final MongoNode masterNode = mongoDbReplicaSet.getMasterMongoNode(
                mongoDbReplicaSet.getMongoRsStatus().getMembers()
            );

            //cut off the primary node from network
            mongoDbReplicaSet.disconnectNodeFromNetwork(masterNode);
            //wait until a new primary is elected that is different from the masterNode
            mongoDbReplicaSet.waitForMasterReelection(masterNode);
            assertThat(
                mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
                hasItems(
                    ReplicaSetMemberState.PRIMARY,
                    ReplicaSetMemberState.ARBITER
                )
            );

            //bring back the disconnected masterNode
            mongoDbReplicaSet.connectNodeToNetwork(masterNode);
            //wait until all nodes are up and running
            mongoDbReplicaSet.waitForAllMongoNodesUp();
            assertThat(
                mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
                hasItems(
                    ReplicaSetMemberState.PRIMARY,
                    ReplicaSetMemberState.ARBITER,
                    ReplicaSetMemberState.SECONDARY
                )
            );
        }
    }
}

See more examples in the test sources mongodb-replica-set on github
See a full Spring Boot + Spring Data example wms on github

Motivation

Cross-platform solution that doesn't depend on fixed ports;
Testing MongoDB transactions to run against an environment close to a production one;
Testing production issues by recreating a real MongoDB replica set (currently without shards);
Education to newcomers to the MongoDB world (learning the behaviour of a distributed NoSQL database while dealing with network partitioning, analyze the election process and so on).

General info

Click to see how to create a 3 node replica set on fixed ports via Docker manually

MongoDB starting from version 4 supports multi-document transactions only on a replica set. For example, to initialize a 3 node replica set on fixed ports via Docker, one has to do the following:

Add 127.0.0.1 mongo1 mongo2 mongo3 to the host file of an operation system;
Run in terminal:
- docker network create mongo-cluster
- docker run --name mongo1 -d --net mongo-cluster -p 50001:50001 mongo:4.0.10 mongod --replSet docker-rs --port 50001
- docker run --name mongo2 -d --net mongo-cluster -p 50002:50002 mongo:4.0.10 mongod --replSet docker-rs --port 50002
- docker run --name mongo3 -d --net mongo-cluster -p 50003:50003 mongo:4.0.10 mongod --replSet docker-rs --port 50003

Prepare the following unix end of lines script (optionally put it folder scripts or use rs.add on each node):

rs.initiate({
    "_id": "docker-rs",
    "members": [
        {"_id": 0, "host": "mongo1:50001"},
        {"_id": 1, "host": "mongo2:50002"},
        {"_id": 2, "host": "mongo3:50003"}
    ]
});

Run in terminal:
- docker cp scripts/ mongo1:/scripts/
- docker exec -it mongo1 /bin/sh -c "mongo --port 50001 < /scripts/init.js"

As we can see, there is a lot of operations to execute and we even didn't touch a non-fixed port approach. That's where the MongoDbReplicaSet might come in handy.

Supported features

Feature	Description	default value	how to set
replicaSetNumber	The number of voting nodes in a replica set including a master one	1	MongoDbReplicaSet.builder()
awaitNodeInitAttempts	The number of approximate seconds to wait for a master or an arbiter node(if addArbiter=true)	29 starting from 0	MongoDBReplicaSet.builder()
propertyFileName	yml file located on the classpath	none	MongoDbReplicaSet.builder()
mongoDockerImageName	a MongoDB docker file name	mongo:4.0.10	finds first set: 1) MongoDbReplicaSet.builder() 2) the system property mongoReplicaSetProperties.mongoDockerImageName 3) propertyFile 4) default value
addArbiter	whether or not to add an arbiter node to a cluster	false	MongoDbReplicaSet.builder()
slaveDelayTimeout	whether or not to create one master and the others as delayed members	false	MongoDbReplicaSet.builder()
useHostDockerInternal	If true then use `host.docker.internal` of Docker, otherwise take `dockerhost` of Qoomon docker-host	false	finds first set: 1) MongoDbReplicaSet.builder() 2) the system property mongoReplicaSetProperties.useHostDockerInternal 3) default value
addToxiproxy	whether or not to create a proxy for each MongoDB node via Toxiproxy	false	MongoDbReplicaSet.builder()
enabled	whether or not MongoReplicaSet is enabled even if instantiated in a test	true	finds first set: 1) the system property mongoReplicaSetProperties.enabled 2) propertyFile 3) default value
commandLineOptions	command line options, example:`Arrays.asList("--oplogSize", "50")`	emptyList	MongoDbReplicaSet.builder()

a propertyFile.yml example:

mongoReplicaSetProperties:
  enabled: false
  mongoDockerImageName: mongo:4.1.13

License

Apache License, Version 2.0

silaev/mongodb-replica-set