Error when starting demo challenge
hmkyriacou opened this issue · 12 comments
I am trying to start the demo pwn challenge but I am getting this error.
The push refers to repository [eu.gcr.io/ctf-infra-test/demo-pwn-chal-challenge] Get "https://eu.gcr.io/v2/": dial tcp [2607:f8b0:400c:c02::52]:443: connect: cannot assign requested address [E] command returned 1
Sometimes I do not get the error, and it seems like it worked, but I still cannot connect to the challenge.
Not sure what it means, or if it is related to not being able to connect to it.
I have been following the kCTF Infrastructure Walkthrough.
Thanks for any help.
Hmm, that is weird. It sounds like you might have run out of local ports?
Does the problem persist after a reboot?
Hmm, that is weird. It sounds like you might have run out of local ports?
Does the problem persist after a reboot?
I was doing this on the Google Cloud Shell.
I was able to connect using the IP address found in the GKE console, but not the domain when the command did not throw that error.
hmm so the DNS is not working?
When i do not get that error, I am able to connect using the IP but I get that error sometimes and the challenge fails to start.
Could that error be connected to the domain not working as well?
I do believe that the "cannot assign requested address" is a problem with the VM itself. I.e. man connect says:
EADDRNOTAVAIL
(Internet domain sockets) The socket referred to by sockfd
had not previously been bound to an address and, upon
attempting to bind it to an ephemeral port, it was
determined that all port numbers in the ephemeral port
range are currently in use. See the discussion of
/proc/sys/net/ipv4/ip_local_port_range in [ip(7)](https://man7.org/linux/man-pages/man7/ip.7.html).
That being said, I don't know why the connection doesn't work in the cases where you don't get an error. Can you check kctf chal status to see if the challenge is running properly?
Here is the output of kctf chal status
cscexec_wpi@cloudshell:~/kctf-test/demo-pwn-chal (ctf-infra-test)$ kCTF[ctf=kctf-test,config=remote-cluster,chal=demo-pwn-chal] > kctf chal status
= CHALLENGE RESOURCE =
NAME HEALTH STATUS DEPLOYED PUBLIC
demo-pwn-chal disabled Running true true
= INSTANCES / PODs =
Challenge execution status
This shows you how many instances of the challenges are running.
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo-pwn-chal-7c87559d56-njp2b 1/1 Running 0 18h 10.48.0.8 gke-kctf-cluster-default-pool-cfe86287-1pbq <none> <none>
demo-pwn-chal-7c87559d56-zttpb 0/1 Terminated 0 42h <none> gke-kctf-cluster-default-pool-cfe86287-sfzh <none> <none>
= DEPLOYMENTS =
Challenge deployment status
This shows you if the challenge was deployed to the cluster.
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
demo-pwn-chal 1/1 1 1 42h challenge eu.gcr.io/ctf-infra-test/demo-pwn-chal-challenge:cf81a5092f8dea7694ea2dd23770a238c841230f9d90e33bba260bddaad817e3 app=demo-pwn-chal
= EXTERNAL SERVICES =
Challenge external status
This shows you if the challenge is exposed externally.
SERVICES:
NAME TYPE EXTERNAL-IP PORT DNS
demo-pwn-chal NodePort <none> 1337 <none>
demo-pwn-chal-lb-service LoadBalancer 34.147.19.31 1337 demo-pwn-chal.wpictf-codelab.kctf.cloud
Ingresses:
No resources found in default namespace.
I don't see anything weird with that.
Here is me trying to connect to it:
cscexec_wpi@cloudshell:~/kctf-test/demo-pwn-chal (ctf-infra-test)$ kCTF[ctf=kctf-test,config=remote-cluster,chal=demo-pwn-chal] > nc demo-pwn-chal.wpictf-codelab.kctf.cloud 1337
nc: getaddrinfo for host "demo-pwn-chal.wpictf-codelab.kctf.cloud" port 1337: Name or service not known
cscexec_wpi@cloudshell:~/kctf-test/demo-pwn-chal (ctf-infra-test)$ kCTF[ctf=kctf-test,config=remote-cluster,chal=demo-pwn-chal] > nc 34.147.19.31 1337
== proof-of-work: disabled ==
CTF{TestFlag}
Is there anything else I should try?
So I have been testing and rebuilding the cluster and challenges and the error I originally posted about has not been showing up.
I am still having DNS issues though, and I am not sure how to further debug it. I have just been following the tutorial, so I set the public field in the yaml file to true and it says it provisioned a DNS record, but I cannot connect using it.
On the DNS console, I see the wpictf-codelab.kctf.cloud record, but I do not see any challenge records. Is there any other options that I need to enable?
Thanks
Edit: I can manually add the record and it works fine.
can you find the DNS daemonset on console.cloud.google.com? it should be under GKE -> Workloads and see its logs
So i see that it the external-dns workload is not working. When I check the logs I see this error:
"pkg/mod/k8s.io/client-go@v0.22.2/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:kctf-system:external-dns-sa" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope"
that should be fixed with 9d1f2a5