TheNewNormal/corectl

Cannot start corectld

Closed this issue · 27 comments

If I try to start the server via corectld start -d I receive the following error.

goroutine 1 [running]:
panic(0x4742240, 0xc820179450)
/usr/local/Cellar/go/1.6.2/libexec/src/runtime/panic.go:481 +0x3e6
github.com/TheNewNormal/corectl/vendor/github.com/helm/helm/log.Die(0xc820054f50, 0x69, 0x0, 0x0, 0x0)
/private/tmp/corectl-20160713-88148-1s3sfsb/corectl-0.7.9/src/github.com/TheNewNormal/corectl/vendor/github.com/helm/helm/log/log.go:46 +0xe1
main.main()
/private/tmp/corectl-20160713-88148-1s3sfsb/corectl-0.7.9/src/github.com/TheNewNormal/corectl/cmd/server/main.go:77 +0x7a

Any ideas on how to solve this?

Hi! some quick questions

a) how did you build the blob / where did you fetched it?
b) macOS version, hw version
c) can you confirm that in current tip issue persists on your setup ?

Thanks in advance!

Hello!

a) Sorry I did not get the question
b) macOS Version is 10.11.5 on an iMac (27, End of 2012, Intel Core i5)
c) I tried 0.7.13 and the problem still exists

goroutine 1 [running]:
panic(0x4b3a380, 0xc8202926e0)
/usr/local/Cellar/go/1.6.3/libexec/src/runtime/panic.go:481 +0x3e6
github.com/TheNewNormal/corectl/vendor/github.com/helm/helm/log.Die(0xc820122ee0, 0x69, 0x0, 0x0, 0x0)
/private/tmp/corectl-20160721-98413-1ugdttd/corectl-0.7.13/src/github.com/TheNewNormal/corectl/vendor/github.com/helm/helm/log/log.go:46 +0xe1
main.main()
/private/tmp/corectl-20160721-98413-1ugdttd/corectl-0.7.13/src/github.com/TheNewNormal/corectl/cmd/server/main.go:77 +0x7a

Thank you for your help!

ok - translating a) how was your binary built ? homebrew ? make from local git checkout ?

I installed it with Homebrew using brew install corectl like mentioned in the Readme.

interesting :/. will get back after your issue after my lunch break

hi again - plz paste here the output of sysctl machdep.cpu.features

thanks in advance and for your patience!

machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC POPCNT AES PCID XSAVE OSXSAVE TSCTMR AVX1.0 RDRAND F16C

Seems like my CPU does not support EPT, right? 😢

kern.hv_support: 1

strange , as it should work then afaict. what timezone are you on ?

CEST / UTC+2

two hrs ahead of me. will ping you monday around your 11.30 am. ok to you ?

Sounds perfect, thank you so much for looking into this!

you're welcome! have a great weekend!

drgeb commented

I am observing the same thing! Thanks!

sysctl machdep.cpu.features
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C
sysctl kern.hv_support
kern.hv_support: 1

I was able to figure out that the file /etc/exports is causing the problem.
So did mv /etc/exports to /etc/exports.bak
Once I moved it to /etc/exports.bak it ran without an issue.
I don't know much about networking or why this is causing an issue.

The earlier entries in the /exports.bak file seem to have been created by vagrant

@drgeb @fantastic-relations

a quick question - did ever corectl run on your setups ?
(trying to spot a pattern here as this is not definitivelly a general issue afaict)

also, can one of you ping me at kubernetes slack please in order to try to debug this in an affected setup ?

I have never been able to use corectl because the server component was missing (Failed to connect to server).
I will ping you on Slack in a minute.

@drgeb:

@fantastic-relations issue was that there was a locally running DNS server (ence binding :53) that was blocking corectld from finishing the startup, so no inherent breackage on our side...

So, can you please ping me too on slack to see what your issue after all :-) ?

@drgeb ping (see above)

I've gotten the same error, because I run dnscrypt. I'd like to continue running dnscrypt, while corectld runs too. Is this possible to tell corectld to skip local DNS server (I'm not sure what that would mean)? Edit: in other words, is it possible to make corectld work like it did before I updated (which was also v7, but not sure which minor version).

Also, the error message includes, “Please […] ensure that 'corectld' is running” which is a little disorienting, because the error prints when trying to run corectld. Here's my full error message (maybe helpful for google to index the other parts of the error):

[q@localhost] corectld start -d
[ERROR] Failed to connect to server
Please check your connection settings and ensure that 'corectld' is running.

panic: Failed to connect to server
Please check your connection settings and ensure that 'corectld' is running.


goroutine 1 [running]:
panic(0x4b3f380, 0xc820191780)
    /usr/local/Cellar/go/1.6.3/libexec/src/runtime/panic.go:481 +0x3e6
github.com/TheNewNormal/corectl/vendor/github.com/helm/helm/log.Die(0xc82011eee0, 0x69, 0x0, 0x0, 0x0)
    /private/tmp/corectl-20160803-63002-1bz84qn/corectl-0.7.15/src/github.com/TheNewNormal/corectl/vendor/github.com/helm/helm/log/log.go:46 +0xe1
main.main()
    /private/tmp/corectl-20160803-63002-1bz84qn/corectl-0.7.15/src/github.com/TheNewNormal/corectl/cmd/server/main.go:77 +0x7a

@quinncomendant valid points. just plz forgive me our latency in handling them as i'll be in vacation til beggining of September with very limited connectivity. will be back to you just after i'm back.

once again many thanks for your patience!

Thanks @AntonioMeireles. In the meantime I've reverted to v0.7.0, which still works fine alongside dnscrypt (this is the last version I was using, saved locally by homebrew; I'm not saying it's the only v0.7.x version that works with local DNS).

Perhaps corectld could check if port 53 is already in use, and if so it continues to load but without DNS and displays a warning message what happened. People running a local DNS server (both dnscrypt and dnsmasq are popular amongst Mac coders) can just use corectld directly without configuring it to ignore DNS, and if they need some corectld DNS feature, they only need to stop their local DNS service first before launching corectld. This is easier than providing a DNS on/off switch, but does rely on displaying a useful warning message so the user knows why corectld DNS was enabled/disabled.

@quinncomendant is this particular issue still a problem for you ? (sorry latency, and have a great weekend)

I've still been using v0.7.0, which works fine. I have been busy and haven't updated for fear that I'd get stuck with the same issue in a newer version. Do newer versions of corectld include an option to skip local DNS server?

@quinncomendant since 632c249 got merged there should be no more collisions with whatever local dns setup is present as (quoting changelog) since then instead of running our embedded name server on host's port 53 we run it on port 15353 and transparently port-forward via pf it as :53 to the VMs. this should handle most, if not all, of the "corner" cases that our user base has been hitting lately...

Please check if this fully covers your use cases :-)

I have updated to version 0.7.18 and it works great. Thank you!

closing per @quinncomendant's latest feedback.