How can I clean everything up after kubeadm reset??

[joungwoongpark]

Hi.
I always appreciate your post.
I finished intergrating with K8S successfully.
But I have a problem after kubectl delete ncp.yml and kubeadm reset.
When I deleted segments, IP Pool and IP Block on the Simplied UI of nsx manager.
I found It could not.
along error message. There is someting what still remained at there.
I guess it is coredns. but as I know
coredns would be cleared after kubeadm reset.
So What is the problem??and How can I delete segments, IP Pool and IP Block??

( I used nsxt 2.4.1, k8s 1.15.3 and NCP 2.5.0 and 2.5.1(both same issue))

[dumlutimuralp]

Thanks for the feedback.

As I understand "kubeadm reset" unconditionally resets the whole K8S cluster. Even if that is NOT the case, if you delete the NCP right away ,(by using kubectl delete ncp.yml), this will not affect the objets which were already provisioned on NSX Manager by NCP earlier on. The reason is think of NCP as just a bridging point between NSX Management Plane and K8S control plane. It is in that sense kind of stateless. Just makes sure that every object in K8S etcd has a corresponding NSX-T construct in NSX Management plane.

What you would need to do is use a cleanup script such as @yasensim' s => https://github.com/yasensim/k8s-lab/blob/master/nsx_policy_cleanup.py

Or next time, before applying kubeadm reset, remove the namespaces and other K8S objects in the K8S cluster through K8S API and then all you need to remove would be the system namespaces and their corresponding constructs in NSX Management plane.

Not sure if VMware is planning to make the deletion process smoother (ie UI based etc.)

Does this answer your question ?

[yasensim]

First you need to delete NCP deployment and than run the script @dumlutimuralp mentioned to delete all orphaned objects for this cluster in NSX. Example below will delete all NSX objects creatd for k8s clustr with name k8s-cl1.
python nsx_policy_cleanup.py --mgr-ip=10.10.10.10 -u admin -p SecurePass1! -c k8s-cl1 --no-warning -r

[joungwoongpark]

Thank you so much. I try to it