[BUG] Do not create a subscription every time

Lines 163 to 171 in be325fa

    
           sub, err := pubsubClient.CreateSubscription(ctx, subID, 
        
           	pubsub.SubscriptionConfig{ 
        
           		Topic:             pubsubClient.Topic(pubsubTopic), 
        
           		ExpirationPolicy:  expiration, 
        
           		RetentionDuration: retention, 
        
           	}) 
        
           if err != nil { 
        
           	return err 
        
           }

The code above creates a subscription every time. This then causes a bunch of one-off subscriptions (as below). Instead we should probably create a fixed subscription ID and use that (doc)

Each subscriber (with a unique ID) can only receives a topic once, is it possible to let all N subscriber with the same ID receive the topic? @pradh

I think you mean: can N subscriber tasks using the same subscription receive the same message on a given topic. Right?

That's not possible, and I see the problem, we essentially want a broadcast message, but that is not possible with the Pub/Sub model.

An alternative is to have fixed per-pod subscribers "mixer-subscriber-pod-PodID", but what we have is probably not too bad given the constraints.

There are ~567 of them. Just wanted to confirm they do get deleted eventually? In the fixed subscriber approach, we'll only have as many subscribers as pods.

The number of pods in website/mixer (prod/staging/autopush/dev) are: [60, 24, 12, 6] and [54, 24, 8]. So during non-deploy time, there could be ~180 subscribers. A subscriber will expires after 36h of inactivity (config).

Note the actual subscribers could be larger than 180 after staging/prod push (almost double it), with cases that pod could restart a few times.

Maybe we can try this to delete the subscription on pod shutdown, so the number could stay around ~180

The subscribers are now deleted when server is shut down: https://github.com/datacommonsorg/mixer/blob/master/cmd/main.go#L156-L164

	sub, err := pubsubClient.CreateSubscription(ctx, subID,
	pubsub.SubscriptionConfig{
	Topic: pubsubClient.Topic(pubsubTopic),
	ExpirationPolicy: expiration,
	RetentionDuration: retention,
	})
	if err != nil {
	return err
	}