Feature request: IPv6 address support
taxilian opened this issue · 11 comments
Kubernetes has added full support for dual stack services; it would be nice if kube-vip-cloud-provider could handle and assign ipv6 addresses as well as ipv4.
see https://kubernetes.io/docs/concepts/services-networking/dual-stack/
There are a couple of ways you could approach this; I think the easiest way would be to look at the clusterIPs array and assign one address for each class of ip that is in the list. I was able to get a semi-hacked version of the MetalLB controller working this way (from metallb/metallb#727), but they don't have it in the mainstream tree yet and it hasn't been updated recently.
Here is an example service from that, with ip addresses tweaked:
apiVersion: v1
kind: Service
metadata:
annotations:
meta.helm.sh/release-name: external
meta.helm.sh/release-namespace: ingress-haproxy
metallb.universe.tf/address-pool: external
creationTimestamp: "2021-09-08T23:58:49Z"
labels:
app.kubernetes.io/instance: external
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: haproxy-ingress
app.kubernetes.io/version: v0.13.3
helm.sh/chart: haproxy-ingress-0.13.3
name: external-haproxy-ingress
namespace: ingress-haproxy
resourceVersion: "223919524"
uid: 7ef512e2-9a16-405b-a5a3-9d9892df5d2b
spec:
allocateLoadBalancerNodePorts: true
clusterIP: 10.96.10.221
clusterIPs:
- 10.96.10.221
- xxxx:xxxx:1000:21::20:f86f
externalTrafficPolicy: Local
healthCheckNodePort: 31319
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
- IPv6
ipFamilyPolicy: PreferDualStack
ports:
- name: http-80
nodePort: 32117
port: 80
protocol: TCP
targetPort: http
- name: https-443
nodePort: 31303
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/instance: external
app.kubernetes.io/name: haproxy-ingress
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: xx.xx.xx.160
- ip: xxxx:xxxx:1000:21::40:0
It's worth noting that the loadBalancerIP field does not support an array of addresses, so there is no way in the spec to request more than one specific ip address -- some people get around that by creating two services (one ipv4, one ipv6), or using a custom annotation, e.g. the metalLB PR I referenced uses metallb.universe.tf/load-balancer-ips: "10.0.0.20,1000::20"
I am available for discussion in the kubernetes slack in the #kube-vip room -- happy to help as I'm able, though I'm super swamped with a bunch of things right now and I'm not super familiar with golang.
Are there any prospects for getting this support? Is it in the roadmap at all? I have an IPv6-only cluster running kube-vip and I'd much rather use this than metalLB, since it sounds like metalLB would be pretty redundant.
I ended up using metallb; if you have the BGP stuff done you can disable part of it, I'm only running the controller.
This project would probably be better for my needs if it supported ipv6, but metallb does have full support for ipv6 now. (it didn't when I opened the ticket)
Are you still using kube-vip with your metalLB setup? If you don't mind me asking, what is your configuration? Which CNI? I have Calico running a purely ipv6 network with BGP between the nodes, and kube-vip running ARP mode.
When you say having BGP set up, is that referring to kube-vip in BGP mode or the CNI?
No, there is no need. I'm using Calico.
There are two pieces to MetalLB -- the controller which assigns IP addresses to LoadBalancer services and the route advertiser which handles BGP connections to advertise the routes out of the cluster.
Calico already uses BGP, so you're better off doing the BGP peering to your ToR router via CalicoBGP and in your Calico "BGPConfiguration" resource you set the serviceExternalIPs and serviceLoadBalancerIPs addresses so that it will advertise those as well.
Kube-VIP and metallb basically do exactly the same thing (both parts) except that Kube-VIP supports the other ARP mode as well, so I can't think of any reason you'd ever use them together.
When you install MetalLB just either only install the controller or else remove the advertiser piece after installation and as long as your calico BGP peering is configured correctly everything should Just Work.
There is another option I just stumbled on -- https://purelb.gitlab.io/docs/
PureLB appears to have supported IPv6 for a lot longer and looks like it may be more flexible than metallb -- I haven't used it, but if I had known about it when I was setting things up I would likely have tried it first as it looks like a better fit for my needs.
Interesting, I'll have to check that out. Maybe it addresses the concern I have regarding metallb.
Regarding metallb,it looks like it doesn't provide the high availability static pod functionality for apiserver (or another alternative) like kube-vip does. Did you get yours to have similar functionality?
Just FYI, I've spotted the activity on this thread. I've started adding IPv6, also there is Gateway API coming which will also provide the support.
For those following this issue, looks like a PR for IPv6 support has been opened in #56.
The current change in #56 only supports single stack.
Will work on dual stack in a bit