Invalid Ingress Definition With IngressRuleValue
dmizelle opened this issue · 4 comments
Hey all,
I'm trying to make a few functions to let developers have an easier time defining Kubernetes resources rather than using YAML, but I'm running into a very strange issue with regards to writing a function for Ingresses:
def ingress(
name,
service=None,
cloudflare=False,
ingress_class="private",
additional_annotations={},
additional_labels={},
services=[], # { "name": "graph", "port": 3000 }
):
i = extensionsv1beta1.Ingress()
i.metadata.name = name
annotations = i.metadata.annotations
if cloudflare:
annotations.update(
{
"certmanager.k8s.io/acme-challenge-type": "http01",
"external-dns.alpha.kubernetes.io/cloudflare-proxied": "true",
}
)
domains = ["example.com"]
else:
annotations.update(
{"certmanager.k8s.io/acme-challenge-type": "dns01",}
)
domains = ["example.com", "example.co.uk", "example.de", "example.ca"]
annotations.update(
{
"certmanager.k8s.io/cluster-issuer": "letsencrypt",
"ingress.kubernetes.io/ssl-redirect": "true",
"kubernetes.io/ingress.class": ingress_class,
"kubernetes.io/tls-acme": "true",
}
)
rules = i.spec.rules
for domain in domains:
rules.append(
extensionsv1beta1.IngressRule(
# issue starts here
ingressRuleValue=extensionsv1beta1.IngressRuleValue(
http=extensionsv1beta1.HTTPIngressRuleValue(
paths=[
extensionsv1beta1.HTTPIngressPath(
path="/",
backend=extensionsv1beta1.IngressBackend(
serviceName=service.metadata.name,
servicePort=intstr.IntOrString(
intVal=service.spec.ports[0].port
),
),
)
]
)
),
host="{}.{}.k8s.{}".format(name, "dev", domain),
)
)
annotations.update(additional_annotations)
return iI've marked above where I'm running into the issue.
The above starlark generates yaml like the following:
- host: nginx.dev.k8s.example.de
ingressRuleValue:
http:
paths:
- path: /
backend:
serviceName: nginx
servicePort: 8081As you can see if you are familiar with Ingress objects, ingressRuleValue isnt supposed to be there, and generates the following error if you try and dry-run apply this with kubectl:
error: error validating "STDIN": error validating data: [ValidationError(Ingress.spec.rules[0]): unknown field "ingressRuleValue" in io.k8s.api.extensions.v1beta1.IngressRule, ValidationError(Ingress.spec.rules[1]): unknown field "ingressRuleValue" in io.k8s.api.extensions.v1beta1.IngressRule, ValidationError(Ingress.spec.rules[2]): unknown field "ingressRuleValue" in io.k8s.api.extensions.v1beta1.IngressRule, ValidationError(Ingress.spec.rules[3]): unknown field "ingressRuleValue" in io.k8s.api.extensions.v1beta1.IngressRule];
The path should have a definition of:
- host: nginx.dev.k8s.example.de
http:
paths:
- path: /
backend:
serviceName: nginx
servicePort: 8081
By taking a look at the documentation of IngressRule, it looks like the field of IngressRuleValue doesn't actually have a name:
https://godoc.org/k8s.io/api/extensions/v1beta1#IngressRule
Is this what could be causing this? Is there a workaround I can use?
Kubernetes has different schemas for inputs in YAML and Protobuf format. Skycfg generates Protobuf, so you'll want to submit the input to the Kubernetes API server as application/vnd.kubernetes.protobuf. The upstream documentation at https://kubernetes.io/docs/reference/using-api/api-concepts/#protobuf-encoding has additional details about the expected encoding.
I'm not sure whether kubectl supports sending requests in Protobuf format -- kubernetes/kubernetes#50403 suggests it doesn't. This may be a blocker if you want to use kubectl inputs as an intermediate format between Skycfg and Kubernetes.
You may be interested in Isopod (https://github.com/cruise-automation/isopod), which uses Skycfg as a base language and layers on Kubernetes-specific behaviors. It may already have functionality to construct kubectl-compliant YAML, or if not, it would be a good place to add that functionality.
👋
I was originally using the example from here:
Lines 321 to 338 in b7cd9ca
Instead, I changed it to look something like this (which is marshalling the proto/struct to JSON, then to YAML) instead of using jsonpb and I ended up getting valid kubectl-friendly YAML!
for _, msg := range protos {
group, version, kind, err := gvkFromProto(msg)
if err != nil {
die("unable to parse group/version/kind from protobuf message", err)
}
marshaled, err := json.Marshal(msg)
if err != nil {
die("unable to marshal struct to json", err)
}
var yamlMap yaml.MapSlice
err = yaml.Unmarshal(
[]byte(marshaled),
&yamlMap,
)
if err != nil {
die("unable to convert json to yaml mapslice", err)
}
yamlMarshaled, err := yaml.Marshal(yamlMap)
if err != nil {
die("unable to generate yaml document from yaml mapslice", err)
}
fmt.Println("---")
apiVersion := ""
if group != "" {
apiVersion = fmt.Sprintf("%s/%s", group, version)
} else {
apiVersion = version
}
fmt.Printf("apiVersion: %s\n", apiVersion)
fmt.Printf("kind: %s\n", kind)
fmt.Println(string(yamlMarshaled))
}
It works for now, so I'll close out this issue. Thanks for a great project.