Unable to connect to new primary when using Replica Set

[noftasmos1998]

Python 3.9.1
PyMongo 4.0.2
Mongoengine 0.24.1

I'm having an issue when trying to connect to a new primary instance. I'm using docker to host the three mongo instances that I have, I'm using this docker-compose to generate said instances:

version: '3.1'

services:
  local1:
    image: bitnami/mongodb:4.2
    ports:
      - 27017:27017
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=local1
      - MONGODB_REPLICA_SET_MODE=primary
      - MONGODB_PRIMARY_HOST=local1
      - MONGODB_REPLICA_SET_KEY=***
      - MONGODB_USERNAME=${MONGODB_USERNAME}
      - MONGODB_PASSWORD=${MONGODB_PASSWORD}
      - MONGODB_DATABASE=${MONGODB_DB}
      - MONGODB_ROOT_PASSWORD=${MONGODB_ROOT_PASSWORD}
    restart: always
    volumes:
      - ./volumes/local1:/bitnami

  local2:
    image: bitnami/mongodb:4.2
    restart: always
    ports:
      - 27018:27017
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=local2
      - MONGODB_REPLICA_SET_MODE=secondary
      - MONGODB_PRIMARY_HOST=local1
      - MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=${MONGODB_ROOT_PASSWORD}
      - MONGODB_REPLICA_SET_KEY=***
      - MONGODB_USERNAME=${MONGODB_USERNAME}
      - MONGODB_PASSWORD=${MONGODB_PASSWORD}
      - MONGODB_DATABASE=${MONGODB_DB}
      - ALLOW_EMPTY_PASSWORD=yes
    depends_on:
      - local1
    volumes:
      - ./volumes/local2:/bitnami

  local3:
    image: bitnami/mongodb:4.2
    restart: always
    ports:
      - 27019:27017
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=local3
      - MONGODB_REPLICA_SET_MODE=secondary
      - MONGODB_PRIMARY_HOST=local1
      - MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=${MONGODB_ROOT_PASSWORD}
      - MONGODB_REPLICA_SET_KEY=***
      - MONGODB_USERNAME=${MONGODB_USERNAME}
      - MONGODB_PASSWORD=${MONGODB_PASSWORD}
      - MONGODB_DATABASE=${MONGODB_DB}
      - ALLOW_EMPTY_PASSWORD=yes
    depends_on:
      - local1
    volumes:
      - ./volumes/local3:/bitnami

When all instances are up, my app connects to the primary just fine but once the primary is down and a new instance becomes the primary, my app is no longer able to connect.

This is the connection that I'm using:
connect(host="mongodb://root:123456@local1:27017,local2:27018,local3:27019/?authSource=admin&replicaSet=replicaset")

This is the error that occurs when mongoengine tries to connect to the new primary:

pymongo.errors.ServerSelectionTimeoutError: local1:27017: [Errno 61] Connection refused,local3:27017: [Errno 61] Connection refused,local2:27017: [Errno 61] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 629dbcc4221475f9b83c59a8, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('local1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('local1:27017: [Errno 61] Connection refused')>, <ServerDescription ('local2', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('local2:27017: [Errno 61] Connection refused')>, <ServerDescription ('local3', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('local3:27017: [Errno 61] Connection refused')>]>

This is not a mongodb issue since the change of primary is being done correctly as you can see in the log bellow local2 is the new primary:

	"members" : [
		{
			"_id" : 0,
			"name" : "local1:27017",
			"health" : 0,
			"state" : 8,
			"stateStr" : "(not reachable/healthy)",
			"uptime" : 0,
			"optime" : {
				"ts" : Timestamp(0, 0),
				"t" : NumberLong(-1)
			},
			"optimeDurable" : {
				"ts" : Timestamp(0, 0),
				"t" : NumberLong(-1)
			},
			"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
			"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
			"lastHeartbeat" : ISODate("2022-06-06T08:42:51.043Z"),
			"lastHeartbeatRecv" : ISODate("2022-06-06T08:03:24.361Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "Received heartbeat from member with the same member ID as ourself: 1",
			"syncingTo" : "",
			"syncSourceHost" : "",
			"syncSourceId" : -1,
			"infoMessage" : "",
			"configVersion" : -1
		},
		{
			"_id" : 1,
			"name" : "local2:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 2491,
			"optime" : {
				"ts" : Timestamp(1654504964, 1),
				"t" : NumberLong(21)
			},
			"optimeDate" : ISODate("2022-06-06T08:42:44Z"),
			"syncingTo" : "",
			"syncSourceHost" : "",
			"syncSourceId" : -1,
			"infoMessage" : "",
			"electionTime" : Timestamp(1654502603, 1),
			"electionDate" : ISODate("2022-06-06T08:03:23Z"),
			"configVersion" : 7,
			"self" : true,
			"lastHeartbeatMessage" : ""
		},
		{
			"_id" : 2,
			"name" : "local3:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 2483,
			"optime" : {
				"ts" : Timestamp(1654504964, 1),
				"t" : NumberLong(21)
			},
			"optimeDurable" : {
				"ts" : Timestamp(1654504964, 1),
				"t" : NumberLong(21)
			},
			"optimeDate" : ISODate("2022-06-06T08:42:44Z"),
			"optimeDurableDate" : ISODate("2022-06-06T08:42:44Z"),
			"lastHeartbeat" : ISODate("2022-06-06T08:42:52.112Z"),
			"lastHeartbeatRecv" : ISODate("2022-06-06T08:42:52.111Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "",
			"syncingTo" : "local2:27017",
			"syncSourceHost" : "local2:27017",
			"syncSourceId" : 1,
			"infoMessage" : "",
			"configVersion" : 7
		}
	]

Hope someone can help me!
Thanks.

[bagerard]

MongoEngine uses pymongo behind the scene and the connection management is fully dedicated to pymongo. Please first check by using a raw pymongo.MongoClient if you can replicate the problem (meaning it is a pymongo issue, not a MongoEngine one). we can reopen once you confirm this