AutoMQ/automq

[BUG] There are 3 nodes in the broker-list, but only 2 node startup commands are generated

funky-eyes opened this issue · 2 comments

Version & Environment

What went wrong?

There are 3 nodes in the broker-list, but only 2 node startup commands are generated

bin/automq-kafka-admin.sh generate-start-command --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --controller-list="192.168.1.111:9093,192.168.1.112:9093,192.168.1.113:9093"  --broker-list="192.168.1.111:9092,192.168.1.112:9092,192.168.1.113:9092"
##############  START CMD LIST ###########
You can copy the command to where your AutoMQ tgz located and run following command to start a AutoMQ kafka server:

Ensure that your compute instance already have JDK17 installed. Execute 'java -version' to check.

------------------------ COPY ME  ------------------
bin/kafka-server-start.sh --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --override process.roles=broker,controller --override node.id=0 --override controller.quorum.voters=0@192.168.1.111:9093,192.168.1.112:9093,192.168.1.113:9093 --override listeners=PLAINTEXT://192.168.1.111:9092,CONTROLLER://192.168.1.111:9093,192.168.1.112:9093,192.168.1.113:9093 --override advertised.listeners=PLAINTEXT://192.168.1.111:9092

bin/kafka-server-start.sh --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --override process.roles=broker --override node.id=1 --override controller.quorum.voters=0@192.168.1.111:9093,192.168.1.112:9093,192.168.1.113:9093 --override listeners=PLAINTEXT://192.168.1.111:9092,192.168.1.112:9092,192.168.1.113:9092 --override advertised.listeners=PLAINTEXT://192.168.1.111:9092,192.168.1.112:9092,192.168.1.113:9092


TIPS: Start controllers first and then the brokers.

What should have happened instead?

How to reproduce the issue?

Additional information

Please attach any relevant logs, backtraces, or metric charts.

    public static ServerGroupConfig genBrokerConfig(String ipPortList,
        ServerGroupConfig controllerGroupConfig) {
        String[] ipPortPairs = ipPortList.split(";");
        List<Integer> nodeIdList = new ArrayList<>();
        Map<Integer, String> listenerMap = new HashMap<>();
        int startIndex = controllerGroupConfig.getNodeIdList().size();
        for (int i = startIndex; i < startIndex + ipPortPairs.length; i++) {
            listenerMap.put(i, "PLAINTEXT://" + ipPortPairs[i - startIndex]);
            nodeIdList.add(i);
        }

        return new ServerGroupConfig(
            nodeIdList,
            controllerGroupConfig.getQuorumVoters(),
            listenerMap, listenerMap);

    }

I'm sorry, when I looked at the documentation, I realized that the delimiter is semicolon-separated instead of the more commonly used comma, which puzzled me.

Then I found another issue, when a node serves as both a broker and a controller, it should not output 6 startup commands, but rather 3, shouldn't it?

 bin/automq-kafka-admin.sh generate-start-command --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --controller-list="192.168.1.111:9093;192.168.1.112:9093;192.168.1.113:9093"  --broker-list="192.168.1.111:9092;192.168.1.112:9092;192.168.1.113:9092"
##############  START CMD LIST ###########
You can copy the command to where your AutoMQ tgz located and run following command to start a AutoMQ kafka server:

Ensure that your compute instance already have JDK17 installed. Execute 'java -version' to check.

------------------------ COPY ME  ------------------
bin/kafka-server-start.sh --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --override process.roles=broker,controller --override node.id=0 --override controller.quorum.voters=0@192.168.1.111:9093,1@192.168.1.112:9093,2@192.168.1.113:9093 --override listeners=PLAINTEXT://192.168.1.111:9092,CONTROLLER://192.168.1.111:9093 --override advertised.listeners=PLAINTEXT://192.168.1.111:9092

bin/kafka-server-start.sh --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --override process.roles=broker,controller --override node.id=1 --override controller.quorum.voters=0@192.168.1.111:9093,1@192.168.1.112:9093,2@192.168.1.113:9093 --override listeners=PLAINTEXT://192.168.1.112:9092,CONTROLLER://192.168.1.112:9093 --override advertised.listeners=PLAINTEXT://192.168.1.112:9092

bin/kafka-server-start.sh --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --override process.roles=broker,controller --override node.id=2 --override controller.quorum.voters=0@192.168.1.111:9093,1@192.168.1.112:9093,2@192.168.1.113:9093 --override listeners=PLAINTEXT://192.168.1.113:9092,CONTROLLER://192.168.1.113:9093 --override advertised.listeners=PLAINTEXT://192.168.1.113:9092

bin/kafka-server-start.sh --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --override process.roles=broker --override node.id=3 --override controller.quorum.voters=0@192.168.1.111:9093,1@192.168.1.112:9093,2@192.168.1.113:9093 --override listeners=PLAINTEXT://192.168.1.111:9092 --override advertised.listeners=PLAINTEXT://192.168.1.111:9092

bin/kafka-server-start.sh --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --override process.roles=broker --override node.id=4 --override controller.quorum.voters=0@192.168.1.111:9093,1@192.168.1.112:9093,2@192.168.1.113:9093 --override listeners=PLAINTEXT://192.168.1.112:9092 --override advertised.listeners=PLAINTEXT://192.168.1.112:9092

bin/kafka-server-start.sh --s3-url="s3://http://minio-tdbackup-offline.xx.xx:9000?s3-access-key=kafka&s3-secret-key=xxxxxx&s3-region=kafka&s3-endpoint-protocol=http&s3-data-bucket=kafka&s3-path-style=true&s3-ops-bucket=kafka&cluster-id=RBus1N3zQGqzgtcDZTbwdQ" --override process.roles=broker --override node.id=5 --override controller.quorum.voters=0@192.168.1.111:9093,1@192.168.1.112:9093,2@192.168.1.113:9093 --override listeners=PLAINTEXT://192.168.1.113:9092 --override advertised.listeners=PLAINTEXT://192.168.1.113:9092