Verify NFS mount fails
Closed this issue · 39 comments
In the newest version (as of today, though ) using the 1.9.0a packages I'm getting a verifying NFS mount error on first run through. Though it works on the second invocation.
Here's the output of docker-machine-nfs default:
[INFO] machine presence ... OK
[INFO] machine running ... OK
[INFO] Lookup mandatory properties ... OK
- Machine IP: 192.168.99.100
- Network ID: vboxnet1
- NFSHost IP: 192.168.99.1
[INFO] Configure NFS ...
!!! Sudo will be necessary for editing /etc/exports !!!
Password:
OK
[INFO] Configure Docker Machine ... OK
[INFO] Restart Docker Machine ... OK
[INFO] Verify NFS mount ... FAIL
Cannot detect the NFS mount :(
Hi @gdemad. I believe I was on the latest pull request. Can you try tearing down your docker-machine, starting a new one and try running again? In my original report I stated:
I'm getting a verifying NFS mount error on first run through. Though it works on the second invocation.
Which based on your message "NFS already mounted". Looks like you are on the second run through.
Hi @box-jhuffaker. Sorry, yes, looks like on the second run.
This issue is reproduce in first run.
Thx.
@gdemad No problem, thanks for confirming!
Few things, I spent the morning debugging this. I somehow got to a point where it eventually wouldn't ever mount the drive on one of my VMs (I cleared /etc/exports and re created and things came together). But generally if I keep restarting the VM it will eventually mount it. (I verify by running the df command like the script). I'm not sure if there's a race between nfsd and restarts or some other issue.
I tried moving the restart into the verify loop, but it didn't fix my problem. Which debunks my theory about restarts.
I should note: I'm on OSX El Capitan
I tried moving the nfsd restart into the verify loop. Still not working.
I tried screwing around. Adding an nfsd update... moving boot local to /opt/bootlocal.sh (which seems to be the preferred location on boot2docker 1.9.1 which docker-machine 0.5.1a seems go come with. Also explicitly cleard ~/.docker/machine/cache to remove the boot2docker.iso and caused a re-download.
Basically I'm shooting blindly at things. I'm happy to try things for people that can actually aim fixes, as my machine is reproing.
I notice this in /var/log/bootlocal.log:
Starting nfs client utilities.
mount: RPC: Remote system error - Connection timed out
mount: mounting 192.168.99.1:/Users on /Users failed: Bad file descriptor
I'm not sure what state my nfsd was when I started the VM. But this verifies that the bootlocal.sh is in the right place.
I try remove docker machine and create new instance. See the same result, after start docker-machine-nfs:
Tested with last release updates. Thx @box-jhuffaker ✅
Possible workaround:
I try open new terminal session and see message:
Error running connection boilerplate: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.100:2376": dial tcp 192.168.99.100:2376: i/o timeout
You can attempt to regenerate them using 'docker-machine regenerate-certs name'.
Be advised that this will trigger a Docker daemon restart which will stop running containers.- docker-machine regenerate-certs dev
- docker-machine-nfs dev
Nice mounted! 🔮
I assume this is just because it's the second run. If you remove the
regenerate certs step and do the process. I bet it will also work.
On Nov 29, 2015 12:02 PM, "Dmitry Schegolihin" notifications@github.com
wrote:
Possible workaround:
I try open new terminal session and see message:
Error running connection boilerplate: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.100:2376": dial tcp 192.168.99.100:2376: i/o timeout
You can attempt to regenerate them using 'docker-machine regenerate-certs name'.
Be advised that this will trigger a Docker daemon restart which will stop running containers.
- docker-machine regenerate-certs dev
- docker-machine-nfs dev
Nice mounted! [image: 🔮]
[image: d632e508f7]
https://cloud.githubusercontent.com/assets/414067/11459302/9114085e-96e4-11e5-99c9-21effc6e61a5.jpg—
Reply to this email directly or view it on GitHub
#22 (comment)
.
Hey guys,
I upgraded to latest docker engine, with latest docker machine, latest virtualbox on OS X El Capitan and it all works without issues ...
$ dm create --driver virtualbox tst
Running pre-create checks...
Creating machine...
(tst) OUT | Creating VirtualBox VM...
(tst) OUT | Creating SSH key...
(tst) OUT | Starting VirtualBox VM...
(tst) OUT | Starting VM...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
To see how to connect Docker to this machine, run: docker-machine env tst$ docker-machine-nfs tst
[INFO] machine presence ... OK
[INFO] machine running ... OK
[INFO] Lookup mandatory properties ... OK
- Machine IP: 192.168.99.101
- Network ID: vboxnet4
- NFSHost IP: 192.168.99.1
[INFO] Configure NFS ...
!!! Sudo will be necessary for editing /etc/exports !!!
Password:
OK
[INFO] Configure Docker Machine ... OK
[INFO] Restart Docker Machine ... OK
[INFO] Verify NFS mount ... OK
--------------------------------------------
The docker-machine 'tst'
is now mounted with NFS!
ENJOY high speed mounts :D
--------------------------------------------Are you sure the docker-machine-nfs script is up to date on your machine?
Really, after upgrade docker engine, see nice work!
➜ ~ docker -v
Docker version 1.9.1, build a34a1d5
➜ ~ docker-machine -v
docker-machine version 0.5.1 (7e8e38e)
@gdemad and it still doesn't work?
Okay great :)
Broken for me...
Macbook pro, retina.
VirtualBox 5.0.10 r104061. I installed the extensions.
jhuffaker@mbp-003138 ~ $ docker -v
Docker version 1.9.1, build a34a1d5
jhuffaker@mbp-003138 ~ $ docker-machine -v
docker-machine version 0.5.1 (7e8e38e)
jhuffaker@mbp-003138 ~ $ curl https://raw.githubusercontent.com/adlogix/docker-machine-nfs/master/docker-machine-nfs.sh | sudo tee /usr/local/bin/docker-machine-nfs > /dev/null && sudo chmod +x /usr/local/bin/docker-machine-nfs
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 9229 100 9229 0 0 11397 0 --:--:-- --:--:-- --:--:-- 11393
jhuffaker@mbp-003138 ~ $ docker-machine create --driver=virtualbox tst && /usr/local/bin/docker-machine-nfs tst
Running pre-create checks...
Creating machine...
(tst) OUT | Creating VirtualBox VM...
(tst) OUT | Creating SSH key...
(tst) OUT | Starting VirtualBox VM...
(tst) OUT | Starting VM...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
To see how to connect Docker to this machine, run: docker-machine env tst
[INFO] Configuration:
- Machine Name: tst
- Shared Folder: /Users
- Force: false
[INFO] machine presence ... OK
[INFO] machine running ... OK
[INFO] Lookup mandatory properties ... OK
- Machine IP: 192.168.99.114
- Network ID: vboxnet1
- NFSHost IP: 192.168.99.1
[INFO] Configure NFS ...
!!! Sudo will be necessary for editing /etc/exports !!!
OK
[INFO] Configure Docker Machine ... OK
[INFO] Restart Docker Machine ... OK
[INFO] Verify NFS mount ... FAIL
Cannot detect the NFS mount :(
Here's /etc/exports:
jhuffaker@mbp-003138 ~ $ cat /etc/exports
"/Users" 192.168.99.112 -alldirs -mapall=563785388:1221986466
"/Users" 192.168.99.113 -alldirs -mapall=563785388:1221986466
/Users 192.168.99.114 -alldirs -mapall=563785388:1221986466
Here's the tst bootlocal log:
jhuffaker@mbp-003138 ~ $ docker-machine ssh tst "cat /var/log/bootlocal.log"
Starting nfs client utilities.
I ran this manually from the command line:
jhuffaker@mbp-003138 ~ $ docker-machine ssh tst "sudo mount -t nfs -o noacl,async 192.168.99.1:/Users /Users"
Just seems to hang.
boot2docker info:
Boot2Docker version 1.9.1, build master : cef800b - Fri Nov 20 19:33:59 UTC 2015
Docker version 1.9.1, build a34a1d5
I'm starting to think my VM just can't reach my host laptop for some reason.
It just takes a while. Both bootlocal.log and my manually run mount command ended with:
jhuffaker@mbp-003138 ~ $ docker-machine ssh tst "cat /var/log/bootlocal.log"
Starting nfs client utilities.
mount: RPC: Remote system error - Connection timed out
mount: mounting 192.168.99.1:/Users on /Users failed: Bad file descriptor
@box-jhuffaker Did you already reboot your mac? When nfsd was hanging this somehow unblocked the situation and never happened again.
Oh I really haven't! I'll throw in a reboot and get back to you.
jhuffaker@mbp-003138 ~ $ uptime
8:12 up 13 days, 21:05, 5 users, load averages: 1.75 1.91 1.96
On Mon, Nov 30, 2015 at 7:11 AM, Toni Van de Voorde <
notifications@github.com> wrote:
@box-jhuffaker https://github.com/box-jhuffaker Did you already reboot
your mac? When nfsd was hanging this somehow unblocked the situation and
never happened again.—
Reply to this email directly or view it on GitHub
#22 (comment)
.
A restart fixed it, thanks for the tip and all of your work on this script. Glad I got a pretty good root cause in the end.
@box-jhuffaker glad it works. Cheers
Rebooting fixed the issue for me as well, @huggsboson.
Glad to hear that, @nhooey.
Reboot not worked for me =/
$ docker-machine create --driver virtualbox dev
...
$ docker-machine-nfs dev
[INFO] Configuration:
- Machine Name: dev
- Shared Folder: /Users
- Force: false
[INFO] machine presence ... OK
[INFO] machine running ... OK
[INFO] Lookup mandatory properties ... OK
- Machine IP: 192.168.99.100
- Network ID: vboxnet0
- NFSHost IP: 192.168.99.1
[INFO] Configure NFS ...
!!! Sudo will be necessary for editing /etc/exports !!!
Password:
The nfsd service does not appear to be running.
Starting the nfsd service
OK
[INFO] Configure Docker Machine ... OK
[INFO] Restart Docker Machine ... OK
[INFO] Verify NFS mount ... FAIL
Cannot detect the NFS mount :(
The docker-machine-nfs script stucks here [INFO] Restart Docker Machine ..., so I check docker-machine ls:
$ docker-machine ls
NAME ACTIVE URL STATE URL SWARM DOCKER ERRORS
dev * virtualbox Running Unknown No IP address found 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:4d:fe:0f brd ff:ff:ff:ff:ff:ff
inet6 fe80::a00:27ff:fe4d:fe0f/64 scope link tentative
valid_lft forever preferred_lft forever
Additional info:
$ cat /etc/exports
/Users 192.168.99.100 -alldirs -mapall=501:20
$ docker-machine --version
docker-machine version 0.5.6, build
$ docker --version
Docker version 1.9.1, build a34a1d5
Second (!) run of docker-machine-nfs with patch from issue #31 (sleep 20 after create bootlocal.sh) seems to work for me.
docker@dev:~$ cat /var/lib/boot2docker/bootlocal.sh
#!/bin/sh
sudo umount /Users
sudo mkdir -p /Users
sudo /usr/local/etc/init.d/nfs-client start
sudo mount -t nfs -o noacl,async 192.168.99.1:/Users /Users
@nenemoianemoi Ok thanks for sharing. I guess I will have to put those 20s until docker-machine answers my question about a potential bug on their side.
Cheers
Just ran in to this issue, the sleep 20 statement mentioned in #31 fixed this. It would be nice to have this as a workaround until docker-machine folks fix the bug.
I also had this issue and the reason was "Block incoming connections" for nfsd in Preferences -> Security & Privacy" -> Firewall -> Firewall Options.... I have also unchecked option "Automatically allow signed software to receive incoming connections" there and I think that because of that OS X automatically blocks incoming connections for all software by setting "Block incoming connections", even signed. So you need to manually select "Allow incoming connections" for each blocked software.
Mmmm.. I still have same issue. Is this supposed to work with the new docker-machine?
docker-machine version 0.13.0, build 9ba6da9
Hi @brahama ,
I've got same machine version and I've got no issues at all ...
$ dm -v
docker-machine version 0.13.0, build 9ba6da9
I ran into this issue today =/
$ docker -v Docker version 18.02.0-ce, build fc4de44
$ docker-machine -v docker-machine version 0.13.0, build 9ba6da9
€dit: I was able to get it running. The /etc/hosts file on the host machine was missing 127.0.0.1 localhosts
Deeplink to solution was: #27 (comment)