spring-cloud/spring-cloud-deployer-kubernetes

Upgrade cronjobs from batch/v1beta to batch/v1

anuragpathak21 opened this issue · 29 comments

anuragpathak21 commented

Version 1.21 of kubernetes promoted cron jobs from batch/v1beta to batch/v1

Upgrade cronjobs from batch/v1beta to use batch/v1

anuragpathak21 commented

@onobc Due to removal of batch/v1beta1 from V1.25 we are unable to use SCDF in our cluster with version 1.25.

onobc commented

@anuragpathak21 - I understand. Sorry, if my comment w/ link (but w/o any context) was confusing. It was meant as an internal note to lookup the latest details of the API. I meant to add some context w/ the link.

We are looking into this issue currently and should have a direction in the next couple days.

markpollack commented

I am waiting to hear back from our commercial customers and we will update you shortly. I'd expect we make the update in the next point release FWIW.

Just out of curiosity, are you aware of the features that are available only in our commercial offering?

https://docs.vmware.com/en/VMware-Spring-Cloud-Data-Flow-for-Kubernetes/1.5/scdf-k8s/GUID-pro-features-index.html

onobc commented

We are unable to make this change in 2.10 as we can not exclude k8s version 1.19,1.20 in SCDF 2.10.

We will make this change in 2.11 and the k8s compatibility will look as follows:

  • 2.11 will support k8s 1.21 - 1.25
  • 2.10 supports k8s < 1.25

The next question is when will 2.11 be available. We are working on it currently and will firm up some milestone dates soon.

  • 👍3
dclangan commented

@onobc What is the best way to get updates regarding timing for 2.11 release?

onobc commented

Hi @dclangan .

This week the team will assign issues and a date to the 2.11.0 milestone.

The best way to "track" is the date on the milestone and the issues still in open status on the milestone.

Somewhat unrelated - but good to know... note that we have:

  • 👍1
ganto commented

Any update on the date for the 2.11.0 release? We are currently stuck rolling out OpenShift updates due to this issue.

onobc commented

Hi @ganto
We got pulled off on other issues last week and did not get a chance to firm up the dates. We should have a date set in the next few days.

Thanks

  • 👀1
onobc commented

We have an estimated release date of 5/17/2023.

  • 👍1
v1bh0r commented

Hello, @onobc, is there a revised ETA on this release?

onobc commented

Hello @v1bh0r , yes the date was moved to 2023-06-20. I mentioned in my reply on the duplicate issue but it is a good thing to callout here as the ETA was previously mentioned above. Thanks.

The fix is in the 2.11.0-SNAPSHOT if you want to try it out.

  • ❤️1
v1bh0r commented

Hello @onobc , we are eagerly awaiting the 2.11 release today. Just in case the release gets delayed, can you point me in the right direction to build the docker image for the 2.11 codebase and deploy it as a part of the cluster deployed using the bitnami charts?

  • 👀2
dhpizza commented

Hi @onobc
this is still not released, correct?
What is the new date to have this available? We are using the bitnami chart, so would need a released 2.11.
Will this spring-cloud/spring-cloud-deployer#378 be in this milestone ?

Thank you 🙏

  • 👍9
cppwfs commented

Sorry for the delay in response.
It will be present in the milestone for 2.11.

  • 👍1
dhpizza commented

Hello @onobc , we are eagerly awaiting the 2.11 release today. Just in case the release gets delayed, can you point me in the right direction to build the docker image for the 2.11 codebase and deploy it as a part of the cluster deployed using the bitnami charts?

Hi @v1bh0r

Did you get that working, i.e. using the 2.11-Snapshot with the bitnami chart?

v1bh0r commented

@dhpizza: We have chosen an unconventional workaround - We are scheduling the jobs are regular k8s cron jobs outside of the SCDF.

  • 👍1
dclangan commented

@onobc Noticed that 2.11.0 no longer has a due date - with the looming EOL for GKE 1.24 being 2 months away, is there a revised ETA for 2.11.0, or perhaps a known stable pre-release snapshot that may be recommended for use now?

Thanks!

onobc commented

Hi @dclangan ,
We removed it as it was not doing much good saying that it was 2 weeks past due. While that is 100% true, it is not helpful for those waiting for it and wanting to know the ETA. We will replace it w/ an accurate ETA today in our team meeting.

Thanks for your patience.

  • 👍1
dbasurco commented

Good morning @onobc ,
Do you have a planned release date? we are coming out of EKS version 1.23 and moving to 1.24 with an EOL date so close doesn't fit very well. Thank you very much!

anuragpathak21 commented

7 months and the fix is still not in. This raise doubts about future support of such issues. Should we start looking at alternative to SCDF?

onobc commented

@dbasurco @anuragpathak21 thanks for your continued patience. The date is hardened and is 2023-09-14. The Dataflow 2.11.0 and Deployer 2.9.0 milestone are updated accordingly.

I understand your frustration @anuragpathak21 and I will explain a bit about the delay. Currently the Deployer and Dataflow releases are dependent. The k8s side was ready to go quite a while ago. The Dataflow side however, had large changes to support Spring Boot 2 and 3 | Spring Cloud Task 2 and 3 | Spring Cloud Batch 4 and 5 workloads on the same server. These changes took longer than expected as we ran into a few snafus along the way. It is now working and we feel the above date is realistic and will be accomplished. Going forward we will either break the Deployer and Dataflow releases up or we will be careful about what we put into a release when there are high priority competing interests in the two pieces.

If this were a critical bug or a critical security issue we would have made sure it was addressed - regardless of the situations described above. This one was (arguably) somewhat of a bug and somewhat of a feature request. It should have not taken this long and we will make sure to plan better going forward.

  • 👍1
dbasurco commented

thank you for the explanation! we will wait until the 14th.

  • 👍1
anuragpathak21 commented

Thanks @dbasurco for the detail explanation. We will wait for the release on 14th.

  • 👍1
onobc commented

@dbasurco @anuragpathak21 we were on track to release today and then we found a bug during staging verification. We had a team meeting to decide if we wanted to fix this before the release - and the vote was yes.

Good news is it should be a straight forward fix and we should be able to release tomorrow 09/15 or at the latest Monday 09/18.

Backing issue: spring-cloud/spring-cloud-dataflow#5467

  • ❤️3
dbasurco commented

Hello @onobc i understand there is some new issues :( . Do you know more information about the new release date? thanks for the effort! Best regards,

rallepalle commented

Hi @onobc we are waiting for the new release 2.11.0 since your announcement in may. Now it should appear on 09/18. Unfortunately, it is currently a critical point for us because in a few days a k8s upgrade is pending and then release 2.10.x is no longer supported! We need therefore a safe statement when release 2.11.0 is published! thanks anyway for your great work!

onobc commented

The release is ready and we are dealing w/ CI infrastructure issues and it should be out today. We have been working around the clock since last Friday to get this to Maven Central. It should be today.

Everyone, the release will be out shortly. This will be the last of the "where is the release" messages I answer on this closed ticket.

Thanks

  • 👍3