aws/glide-for-redis

Flakey failure of the cluster script in tests

shachlanAmazon opened this issue · 1 comments

https://github.com/aws/glide-for-redis/actions/runs/7961591871/job/21733197017?pr=971
All the node cluster tests fail, seemingly due to the cluster manager failure.

  console.log
    python3 ../utils/cluster_manager.py start --cluster-mode -r  0 -n 3

      at tests/TestUtilities.ts:197:21

  console.error
    INFO:root:## Executing cluster_manager.py with the following args:
      Namespace(host='127.0.0.1', tls=False, auth=None, log='info', logfile=None, action='start', cluster_mode=True, folder_path='/home/runner/work/glide-for-redis/glide-for-redis/utils/clusters', ports=None, shard_count=3, replica_count=0, prefix='redis-cluster', load_module=None)
    INFO:root:2024-02-19 15:09:46.923248+00:00 Starting script for cluster /home/runner/work/glide-for-redis/glide-for-redis/utils/clusters/redis-cluster-2024-02-19T15-09-46Z-rcl2Ni
    Traceback (most recent call last):
      File "/home/runner/work/glide-for-redis/glide-for-redis/node/../utils/cluster_manager.py", line 1036, in <module>
        main()
      File "/home/runner/work/glide-for-redis/glide-for-redis/node/../utils/cluster_manager.py", line 973, in main
        servers = create_servers(
      File "/home/runner/work/glide-for-redis/glide-for-redis/node/../utils/cluster_manager.py", line 372, in create_servers
        if is_address_already_in_use(server, f"{node_folder}/redis.log"):
      File "/home/runner/work/glide-for-redis/glide-for-redis/node/../utils/cluster_manager.py", line 627, in is_address_already_in_use
        with open(log_file, "r") as f:
    FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/glide-for-redis/glide-for-redis/utils/clusters/redis-cluster-2024-02-19T15-09-46Z-rcl2Ni/25032/redis.log'

      198 |             exec(command, (error, stdout, stderr) => {
      199 |                 if (error) {
    > 200 |                     console.error(stderr);
          |                             ^
      201 |                     reject(error);
      202 |                 } else {
      203 |                     const { clusterFolder, ports } = this.parseOutput(stdout);

Closing - waiting for repro