Multi node deployment with NetworkTopologyStrategy

[vvaldersteins]

So I'm trying to setup multi-node cassandra cluster.

I've setup 3 cassandra nodes on 3 different servers. In all of the three cassandra.yaml files, I've defined the same one seed node. After clearing the data and restarting the services one by one, I can see them showing up and the nodetool status shows UN for all entries. Now I try to do find queries through the API, and that works just fine, but it only works for the first few queries and afterwards, it just crashes with the following error -

Error during find query on DB -> ResponseError: Server failure during read query at consistency ONE (1 responses were required but only 0 replicas responded, 2 failed)

It does not matter on which read query it fails, but it's up for like 10 seconds and then it dies. Sometimes it dies on queries which were successful on previous run. Also, I'm just wondering, why it says that 2 failed, if there are 3 nodes (one seed node)?

I noticed, that express-cassandra notified that the Replication factor has been changed and to run the nodetool repair, but there I keep getting the following error message -

[2018-07-02 08:10:33,447] Some repair failed

Where xxx.xxx.xxx.xxx is one of the three node IP addresses. I tried to run the nodetool repair on each of the nodes, but I keep getting identical errors on each of the servers.

These are my express-cassandra setup properties (xxx.xxx.xxx.xxx is my seed ip address):

    clientOptions: {
      contactPoints: ['xxx.xxx.xxx.xxx'],
      protocolOptions: {
        port: 9042
      },
      keyspace: 'test_keyspace',
      queryOptions: {
        consistency: cassandra.consistencies.one
      },
      socketOptions: {
        readTimeout: 0
      }
    },
    ormOptions: {
      defaultReplicationStrategy: {
        class: 'NetworkTopologyStrategy',
        dc1: 3
      }
    }

I'm not sure if the NetworkTOpologyStrategy is correctly set up, but after starting up the express.js I ran a write query to the database, and it populated the data on all three nodes.

Any help on resolving these both errors would be wonderful!

[masumsoft]

@vvaldersteins Did you try nodetool repair -full?

It looks like this is more of a deployment configuration issue rather than an issue with this module. What does your cassandra-topoplogy.properties and cassandra-rackdc.properties files look like? What is your endpoint_snitch value in cassandra.yaml?