Periscope consumption strategy for new deployment file.
Closed this issue · 8 comments
Periscope consumption strategy.
This issue is created to discuss aspects of key consuming tools which takes deployment file from periscope tool and deploy it, key ideas is to lock in the contract of usage for these consuming tools moving forward.
This discussion will also help in making sure that next release goes out smoothly.
Recently there is some good amount of work done to clean this repo and with the new changes to remove much of clutter and complexity there is decent amount of work went in to manage the yaml file, and using the kustomize concept.
This is again one of the honest appraisal of the situation between both tools: az-cli and vscode which consumes this tool. This discussion will help some of the key maintainers to voice their thoughts and the contract with existing tool which consume this tool. Discussion will also help in making it clear that if this tool co-exist with 2 different eco-system as coupled or totally de-coupled.
Latest clean up to Kustomize yaml file:
One of the key clean up took place was to handle declarative management of the one large *.yaml file into small more readable, and more consumable form. example: https://github.com/Azure/aks-periscope/blob/master/deployment/kustomization.yaml
Current consuming tool contract by use:
Currently in a prod, periscope tool utilises the aks-perisceop.yaml and with consuming tool (like az-cli, and vscode simply replace the storage name and sas key like in az-cli in line 118 https://github.com/Azure/aks-periscope/blob/master/deployment/aks-periscope.yaml#L118 and under the hood they run kubectl apply -f aks-persiceop.yaml to run this tool.
Essentially, I see as a viable solution because landing to the storage is one of the key aspect of this tool.
However we should lock in the usability contract for how we should use kustomize yaml moving forward:
Solutions:
Non-Breaking vs non-breaking vs dependent
There are 2 ways we could use it:
Option 1: Periscope repo owns this deployment contract and holds the deployment file. (Non-Breaking)
This is what the current existing behaviour is. So even with the use of Kustomize file we continue with this structure i.e. make aks-periscope.yaml out of the kustomize.yaml file and maintain at periscope repository. This way I believe do not change the consumption metrics for the az-cli and vscode.
Keep the consuming tool yaml under one directory structure but with in periscope repository. So, even though we are using kustomize file, we can keep equivalent to the aks-perisceop.yaml file within the periscope repo. For the more structural reasons we could move it to deployment/storage/aks-periscope.yaml
For example, in az-cli the collect command use direct deployment file to kubectl apply https://github.com/Azure/azure-cli-extensions/blob/main/src/aks-preview/azext_aks_preview/custom.py#L2073
Option 2: Consuming Tools handle the deployment file. (Breaking - so need consuming tool Co-ordination)
Under this option, the idea is that consuming tool will utilize the the existing kustomize file and maintain their own repository level deployment.yaml file. This will involve both az-cli and vscode to handle the kustomize file handling, it could either mean that consuming tool can do it using: kustomize build … or Kubectl kustomize ….kustmizedeployment > file.yaml and handle deployment. As discussed this will change the consumption metrics for the az-cli and vscode.
This will require changes both of these consuming tool, this also mean that we need careful planning.
For visibility and thoughts: @qpetraroia, @justindavies , @palma21 , @rzhang628
cc: @arnaud-tincelin, @davidkydd , @JunSun17
Thank you so much 🙏❤️
^Tats
Update 8th July:
Option 3: Dependent release Solution: Moving forward solution. (Respecting / Keeping Consuming Metrics Contract Intact for a time being)
(This solution focuses on the fact that other tool need transition time to port new changes)
We need transition for both consuming tools to adapt new changes, I also get a feeling from PM(s) that this tool co-exist with both az-cli and vscode i.e. the solution contract between 2 needs to stay but it is doable without breaking the change will be to respect the contract both tool have with aks-periscope.yaml and then once vscode and az-cli moved to new kustomize.yaml way we remove the old aks-periscope.yaml file in few months (with enough warning) - I believe az-cli release don't happen often and usability is restrictive.
One of the best solution is as follows:
- Keep kustomize solution, but also respect
aks-periscope.yamlfor next 3 - 4 months (Whatever is the best possible transition time for user to adapt) I think there are telemetry for it, for consuming Metrics. - For underlying tools:
- Consuming tools need to do 2 extra steps, to consume
kustomizemechanism.- 1 -
kubectl kustomize -f kustomizefile > localmaintained-aks-pericope-equivalent.yaml - 2 - Add storage to it: https://github.com/Azure/aks-periscope/blob/master/deployment/examples/kustomization_storage_account.yaml
- OR - add we could add Storage to Kustomize ? (This will make it simpler)
- 3 -
kubectl apply -f localmaintained-aks-pericope-equivalent.yaml
- 1 -
- Consuming tools need to do 2 extra steps, to consume
- Once both tool does the above transition and we do not have any other dependency on the
aks-periscope.yamlwe remove it with proper communication in few month time.
Update 21st July
In current state of this repo the only backward compatibility changes we will need in the aks-periscope.yaml are as follows:
Thanks, 🙏
I think (1) is actually a sub-optimal option compared to (call it Option 1a) updating the consuming tools to use "kubectl apply -k" and targeting the kustomization.yaml from the periscope repo. Note this would require first downloading the full directory of kustomize files rather than just targeting the single url; or maybe we can do something with all the "resources:" referencing each other out of github raw itself, rather than assuming it is run from inside a fully checked out periscope repo?
The reason I consider (1) sub-optimal is that we would need to ensure the monolithic .yaml is kept in sync with the Kustomize files, and we run risk that someone will inevitably forget to update the monolithic deployment yaml or update it incorrectly when updating the kustomize files. We could add a post-commit action or similar to automatically build the monolithic deployment files from the Kustomize files and continue to host them out of the periscope repo, but Option 1a seems a cleaner approach.
Both (1) and (1a) would maintain the current situation which I consider dangerous - where there is silent and non-obvious coupling between the consuming tools and the code in this repo. For either (1) or (1a) we should look at adding an integration test running prior to check-in which validates that the consuming tools can run successfully with the new version of the deployment yaml. This would make the coupling obvious to anyone working in Periscope repo, as well as removing the error prone and burdensome need to manually validate changes.
Note the key benefit provided by (1, 1a) over (2) is that we can update the version of periscope deployed by consuming tools without having to update the consuming tools themselves. This is a benefit I think we should look to retain.
Clarification regarding: For either (1) or (1a) we should look at adding an integration test running prior to check-in which validates that the consuming tools can run successfully with the new version of the deployment yaml. This test would ideally build and run the actual consuming tools themselves, but this might not be feasible. Instead we could codify the behavioural contract for what the consuming tools require periscope to do (e.g. storage output format, CRD contents, any requirements on aspects of the config yaml), we can then assert that any changes to core periscope will continue to work with the consuming tools.
Aren't we trying to re-invent a way to do deployment?
If the current kustomize file does not fit our need we can update it or is it the way kustomize work the issue? Do you think Helm is a better option?
Any tool wanting to use aks-periscope will want it to deploy it by a packaging tool. I pushed Kustomize because I believe from experience it's the best but I am open to discussion.
Aren't we trying to re-invent a way to do deployment?
If the current kustomize file does not fit our need we can update it or is it the way kustomize work the issue? Do you think Helm is a better option?
Any tool wanting to use aks-periscope will want it to deploy it by a packaging tool. I pushed Kustomize because I believe from experience it's the best but I am open to discussion.
📓 Kustomize is awesome/great refactor (moving forward we are keeping that approach). (Thank you for that PR and idea btw)
💡 This issue is opened for making sure how az-cli and vscode can keep supporting / have backward compatibility with aks-periscope.yaml deployment for few months (i.e. at-least until consuming tools have moved to new way and have tool transition time). So this workitem / issue is to make sure we don't break that working-contract.
Further explanation: The idea focuses on strategy how not to introduce breaking changes into the consuming tool and Instead have 3-4 month of transition time for both az-cli and vscode to keep working with old deprecated aks-periscope.yaml
With new changes there will be associated changes needed in both az-cli and vscode, and this issue is reference point for those work item in these respective tools to work on. i.e. instead of deploying kubrctl apply -f link_to_raw_aks-periscope.yaml it will become what is explained as Option-3 above.. Further, this issue is also a reminder for periscope repo to archive when this contract was changed and measure we took for other tools to make their changes without effecting current customers using this functionality via az-cli or vscode.
Especially, this tool from day-0 supported az aks kollect command which is az-cli environment if we follow the transition approach we will respect the consuming metrics and give enough time for the developers in other consuming tools to make those changes, and have transition time to deprecate those changes.
Note Azure Arc has the same situation, they have an azure CLI integration for periscope based on the AKS CLI integration and do not have scope in v1 to change deployment method to use kustomize. Will be using a monolithic single deployment yaml initially and then migrating to use kustomize later
Option 3 sounds good to me Tats. I agree that it might be best to first add the storage settings directly to the kustomize files, that way there is one less step for each consuming tool to perform. What is the benefit in keeping this as a separate patch step @arnaud-tincelin?
Update 21st July
In current state of this repo the only backward compatibility changes we will need in the aks-periscope.yaml are as follows:
Thanks, 🙏