Queue Shrinking

[dialtonetech]

Hi,

After running the Cluster example, and loading a number of records into the queue without consumers running, I found that the queue empties.

How do you push to a Queue and only remove once consumed.

Thanks

[mhelvacikoylu]

Hi,

How do you run these sample applications? Debugging producer and consumer applications are ok. But running the server should be different than debugging. Because code is written for 3 instances example. If you just run the project, only one instance will run on port 26101. And producer does not connect to 26101.

So why? We wanted to reproduce some scenarios. Producer has 3 endpoints defined. 2 of them running server ports 26102 and 26103. But other port is random different port. The purpose here, showing that the producer tries to connect another available server. If one of them is not available.

But by default if you debug the server, it will be run only as one instance with 26101. So, producer will never connect to the server and queues will be empty always. You can just add port 26101 to producer. After some connection tries, producer will find the available server and start to produce the messages and you will able to consume them with consumer application.

If you wanna try 3 instance example for server. You can build the server and create 3 different folders. Run each with CLI parameters like this: (For each line, first parameter the server port, other two parameters are other node ports.)

dotnet ClusteringSample.Server.dll 26101 26102 26103
dotnet ClusteringSample.Server.dll 26102 26101 26103
dotnet ClusteringSample.Server.dll 26103 26101 26102

Servers will connect each other and they will choose a master node with leader election algorithm. In this example even the leader is running on port 26101, other servers will tell to the producer the leader is running on 26101 and producer will be able to connect that server by receiving the host information from other servers.

It's possible to run 3 different instances on same server application by using some async operations but the console log might be confusing. You can just to it like this:

Change your main method codes as:

        static void Main(string[] args)
        {
            if (args.Length < 3)
            {
                Task.Run(() => StartServer(26101, 26102, 26103));
                Task.Run(() => StartServer(26102, 26101, 26103));
                Task.Run(() => StartServer(26103, 26101, 26102));
            }
            else
                StartServer(Convert.ToInt32(args[0]),
                    Convert.ToInt32(args[1]),
                    Convert.ToInt32(args[2]));

            Console.ReadLine();
        }

And it's not over, because queue filenames are same and different servers should not use same queue files. So you need to change UsePersistentQueues line as

                    q.UsePersistentQueues(c =>
                    {
                        c.SetPhysicalPath(qn => $"{qn.Name}-server{port}.tdb");
                        c.UseInstantFlush();
                    });

I tested with that way now, it seems everything working well.