DNS lookup fail in guest on centos
ianmiell opened this issue · 7 comments
- checked troubleshooting steps (can ping google dns from host (8.8.8.8))
- checked landrush is up on the host (restart before vagrant up)
imiell@caravaggio:/space/git/shutit-openshift-cluster/vagrant_run/20180426164155 ⑂ master + $ vagrant up master1
Bringing machine 'master1' up with 'libvirt' provider...
==> master1: Creating image (snapshot of base box volume).
[....]
==> master1: [landrush] network: :forwarded_port, {:guest=>22, :host=>2222, :host_ip=>"127.0.0.1", :id=>"ssh", :auto_correct=>true, :protocol=>"tcp"}
[landrush] Using eth0 (192.168.121.197)
==> master1: [landrush] adding machine entry: master1.vagrant.test => 192.168.121.197
==> master1: [landrush] starting dns server
master1: [landrush] 'ruby /home/imiell/.vagrant.d/gems/2.4.4/gems/landrush-1.2.0/lib/landrush/start_server.rb 10053 /home/imiell/.vagrant.d/data/landrush /home/imiell/.vagrant.d/gems/2.4.4/gems'
[landrush] Using eth0 (192.168.121.197)
[landrush] Host dnsmasq config looks good.
logging in:
$ vagrant ssh master1
[vagrant@master1 ~]$ ping google.com
ping: google.com: Name or service not known
$ vagrant landrush status
Daemon status: running pid=11614
$ vagrant landrush vms
- master1.vagrant.test
$ vagrant landrush ls
master1.vagrant.test 192.168.121.197
197.121.168.192.in-addr.arpa master1.vagrant.test
$ ps -ef | grep dnsmas
nobody 4718 1 0 Apr25 ? 00:00:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
root 4719 4718 0 Apr25 ? 00:00:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
imiell 11964 2837 0 18:03 pts/5 00:00:00 grep --color=auto dnsmas
nobody 26559 1 0 15:57 ? 00:00:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/vagrant-libvirt.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
root 26560 26559 0 15:57 ? 00:00:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/vagrant-libvirt.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
Not sure how to debug from here...?
Same issue here : working on Windows 7, vagrant 2.1.1, virtualbox 5.2.12r122591, box centos/7 : enabling landrush break the DNS, my shell provisionning break on yum unable to find mirrors.
@ianmiell is this still a re-producible problem for you? What are the steps to re-produce? Are you saying the problem is a CentOS host?
Which box and config are you using?
If memory serves I had the problem on ubuntu bionic (definitely) and centos/7 (I think), running on a base of ubuntu 18.04 (I think).
I'll try and check it tomorrow.
imiell@cage:~/tmp $ cat Vagrantfile
Vagrant.configure("2") do |config|
config.landrush.enabled = true
config.vm.define "chefserver" do |chefserver|
chefserver.vm.box = "ubuntu/bionic64"
end
end
imiell@cage:~/tmp $ uname -a
Linux cage 4.15.0-38-generic #41-Ubuntu SMP Wed Oct 10 10:59:38 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
imiell@cage:~/tmp $ vagrant version
Installed Version: 2.2.0
Latest Version: 2.2.2
To upgrade to the latest version, visit the downloads page and
download and install the latest version of Vagrant from the URL
below:
https://www.vagrantup.com/downloads.html
If you're curious what changed in the latest release, view the
CHANGELOG below:
https://github.com/hashicorp/vagrant/blob/v2.2.2/CHANGELOG.md
repros the problem:
vagrant@ubuntu-bionic:~$ sudo su
root@ubuntu-bionic:/home/vagrant# apt update
Err:1 http://archive.ubuntu.com/ubuntu bionic InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:2 http://security.ubuntu.com/ubuntu bionic-security InRelease
Temporary failure resolving 'security.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Temporary failure resolving 'archive.ubuntu.com'
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-backports/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
centos/7 looks ok.
We've been running into the same issue on bento/ubuntu-18.04. It seems to be related to a systemd + caching + name resolution failure bug discussed here:
https://moss.sh/name-resolution-issue-systemd-resolved/
By running this script in our Vagrantfile, we're able to prevent issues with the DNS resolution, but unfortunately this causes Landrush to fail when attempting to halt and then re-up a server (vagrant halt && vagrant up). This makes our DNS stable and prevents lookup failures on the initial provisioning & boot, but unfortunately prevents a second "vagrant up" from running.
# update_resolver.sh
sudo rm -f /etc/resolv.conf
sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf
ls -l /etc/resolv.conf
Relevant block from our Vagrantfile:
config.vm.provision :shell,
path: "./vagrant/scripts/update_resolver.sh",
run: "always"
config.landrush.enabled = true
config.landrush.upstream "8.8.8.8"
config.landrush.tld = "test"
config.landrush.host_ip_address = '192.168.99.99'
Here's where our vagrant up breaks on a second attempt:
==> default: Checking for guest additions in VM...
==> default: Configuring and enabling network interfaces...
==> default: Automatic installation for Landrush IP not enabled
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!
cat /etc/resolv.conf | grep ^nameserver
Stdout from the command:
Stderr from the command:
Running Vagrant 2.2.3, bento/ubuntu-18.04, landrush 1.3.1.
Hope this helps with further information/debugging.