spring-cloud/spring-cloud-deployer

Duplicate Kubernetes pod ids when tasks are launched at the same time

sravaniv-3cortex opened this issue · 3 comments

When the Spring Cloud DataFlow Server in the Kubernetes environment tries to launch pods for many tasks(200+), we have encountered scenarios where duplicate Pod id is generated for 3-4 pods. I have looked at the code generating the id, and it encodes the timestamp in milliseconds using the hashid library.

Multiple tasks end up launched simultaneously and with the same pod id. Can we introduce randomness here or another way to ensure pod id uniqueness is not dependent on only the timestamp?

Observed the same behavior. Currently, we work around by catching the exception and trying to re-launch the desired task once, but a fix in Deployer would be nicer.

@onobc I suggest we create a random number within a syncronized block or Lock instead of using millisecond timer.

onobc commented

@onobc I suggest we create a random number within a syncronized block or Lock instead of using millisecond timer.

We can do a concat of the millis + counter suffix (in sync fashion)