Error: Config key "source.wipe" may not be used as member-specific key
Closed this issue · 2 comments
nishit93-hub commented
Hi All,
I am trying to set up microcloud. However, i am getting the error
Error: Config key "source.wipe" may not be used as member-specific key
Below are the steps/ commands I performed.
microcloud@microcloud:~$ sudo snap install lxd
lxd 5.19-8635f82 from Canonical✓ installed
microcloud@microcloud:~$ lxd init
Would you like to use LXD clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]:
Name of the new storage pool [default=default]:
Name of the storage backend to use (zfs, btrfs, ceph, dir, lvm) [default=zfs]:
Create a new ZFS pool? (yes/no) [default=yes]:
Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]:
Size in GiB of the new loop device (1GiB minimum) [default=19GiB]: 40GiB
Would you like to connect to a MAAS server? (yes/no) [default=no]:
Would you like to create a new local network bridge? (yes/no) [default=yes]:
What should the new bridge be called? [default=lxdbr0]:
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
Would you like the LXD server to be available over the network? (yes/no) [default=no]: yes
Address to bind LXD to (not including port) [default=all]:
Port to bind LXD to [default=8443]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]:
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:
microcloud@microcloud:~$ lxc storage create disks zfs size=100GiB
To start your first container, try: lxc launch ubuntu:22.04
Or for a virtual machine: lxc launch ubuntu:22.04 --vm
Storage pool disks created
microcloud@microcloud:~$ **lxc storage set disks volume.size 10GiB**
microcloud@microcloud:~$ lxc storage volume create disks local1 --type block
lxc storage volume create disks local2 --type block
lxc storage volume create disks local3 --type block
lxc storage volume create disks local4 --type block
Storage volume local1 created
Storage volume local2 created
Storage volume local3 created
Storage volume local4 created
microcloud@microcloud:~$ lxc storage volume create disks remote1 --type block size=20GiB
lxc storage volume create disks remote2 --type block size=20GiB
lxc storage volume create disks remote3 --type block size=20GiB
Storage volume remote1 created
Storage volume remote2 created
Storage volume remote3 created
microcloud@microcloud:~$ lxc network create microbr0
Network microbr0 created
microcloud@microcloud:~$ lxc network get microbr0 ipv4.address
lxc network get microbr0 ipv6.address
10.166.166.1/24
fd42:fc85:20e1:ccb5::1/64
microcloud@microcloud:~$ lxc init ubuntu:22.04 micro1 --vm --config limits.cpu=2 --config limits.memory=2GiB
lxc init ubuntu:22.04 micro2 --vm --config limits.cpu=2 --config limits.memory=2GiB
lxc init ubuntu:22.04 micro3 --vm --config limits.cpu=2 --config limits.memory=2GiB
lxc init ubuntu:22.04 micro4 --vm --config limits.cpu=2 --config limits.memory=2GiB
Creating micro1
Creating micro2
Creating micro3
Creating micro4
microcloud@microcloud:~$ lxc storage volume attach disks local1 micro1
lxc storage volume attach disks local2 micro2
lxc storage volume attach disks local3 micro3
lxc storage volume attach disks local4 micro4
lxc storage volume attach disks remote1 micro1
lxc storage volume attach disks remote2 micro2
lxc storage volume attach disks remote3 micro3
microcloud@microcloud:~$ lxc config device add micro1 eth1 nic network=microbr0 name=eth1
lxc config device add micro2 eth1 nic network=microbr0 name=eth1
lxc config device add micro3 eth1 nic network=microbr0 name=eth1
lxc config device add micro4 eth1 nic network=microbr0 name=eth1
Device eth1 added to micro1
Device eth1 added to micro2
Device eth1 added to micro3
Device eth1 added to micro4
microcloud@microcloud:~$ lxc start micro1
lxc start micro2
lxc start micro3
lxc start micro4
microcloud@microcloud:~$ lxc exec micro1 -- bash
root@micro1:~# lxc storage list
If this is your first time running LXD on this machine, you should also run: lxd init
To start your first container, try: lxc launch ubuntu:22.04
Or for a virtual machine: lxc launch ubuntu:22.04 --vm
+------+--------+--------+-------------+---------+-------+
| NAME | DRIVER | SOURCE | DESCRIPTION | USED BY | STATE |
+------+--------+--------+-------------+---------+-------+
root@micro1:~# echo 0 > /proc/sys/net/ipv6/conf/enp6s0/accept_ra
root@micro1:~# ip link set enp6s0 up
root@micro1:~# snap install microceph microovn microcloud --cohort="+"
microceph (quincy/stable) 0+git.7b5672b from Canonical✓ installed
microcloud 1.1-04a1c49 from Canonical✓ installed
microovn (22.03/stable) 22.03.3+snapd29c175b11 from Canonical✓ installed
root@micro1:~#
exit
microcloud@microcloud:~$ lxc exec micro2 -- bash
root@micro2:~# echo 0 > /proc/sys/net/ipv6/conf/enp6s0/accept_ra
root@micro2:~# ip link set enp6s0 up
root@micro2:~# snap install microceph microovn microcloud --cohort="+"
microceph (quincy/stable) 0+git.7b5672b from Canonical✓ installed
microcloud 1.1-04a1c49 from Canonical✓ installed
microovn (22.03/stable) 22.03.3+snapd29c175b11 from Canonical✓ installed
root@micro2:~#
exit
microcloud@microcloud:~$ lxc exec micro3 -- bash
root@micro3:~# echo 0 > /proc/sys/net/ipv6/conf/enp6s0/accept_ra
root@micro3:~# ip link set enp6s0 up
root@micro3:~# snap install microceph microovn microcloud --cohort="+"
microceph (quincy/stable) 0+git.7b5672b from Canonical✓ installed
microcloud 1.1-04a1c49 from Canonical✓ installed
microovn (22.03/stable) 22.03.3+snapd29c175b11 from Canonical✓ installed
root@micro3:~#
exit
microcloud@microcloud:~$ lxc exec micro4 -- bash
root@micro4:~# echo 0 > /proc/sys/net/ipv6/conf/enp6s0/accept_ra
root@micro4:~# ip link set enp6s0 up
root@micro4:~# snap install microceph microovn microcloud --cohort="+"
Download snap "microceph" (707) from channel "quincy/stable" 2% 27.8kB/s 55.8m^C
error: cannot perform the following tasks:
- Download snap "microceph" (707) from channel "quincy/stable" (the download has been cancelled: context canceled)
root@micro4:~#
root@micro4:~#
root@micro4:~#
root@micro4:~# snap install microceph microovn microcloud --cohort="+"
Download snap "microceph" (707) from channel "quincy/stable" 3% 13.9kB/s 1h50m^C
error: cannot perform the following tasks:
- Download snap "microceph" (707) from channel "quincy/stable" (the download has been cancelled: context canceled)
root@micro4:~# snap install microceph microovn microcloud --cohort="+"
microceph (quincy/stable) 0+git.7b5672b from Canonical✓ installed
microcloud 1.1-04a1c49 from Canonical✓ installed
microovn (22.03/stable) 22.03.3+snapd29c175b11 from Canonical✓ installed
root@micro4:~#
exit
microcloud@microcloud:~$ lxc exec micro1 -- bash
root@micro1:~# microcloud init
Waiting for LXD to start...
Select an address for MicroCloud's internal traffic:
Using address "10.191.137.88" for MicroCloud
Limit search for other MicroCloud servers to 10.191.137.88/24? (yes/no) [default=yes]: yes
Scanning for eligible servers ...
Selected "micro1" at "10.191.137.88"
Selected "micro3" at "10.191.137.56"
Selected "micro2" at "10.191.137.48"
Selected "micro4" at "10.191.137.144"
Would you like to set up local storage? (yes/no) [default=yes]: yes
Select exactly one disk from each cluster member:
Select which disks to wipe:
Using "/dev/disk/by-id/scsi-SQEMU_QEMU_HARDDISK_lxd_local1" on "micro1" for local storage pool
Using "/dev/disk/by-id/scsi-SQEMU_QEMU_HARDDISK_lxd_local2" on "micro2" for local storage pool
Using "/dev/disk/by-id/scsi-SQEMU_QEMU_HARDDISK_lxd_local3" on "micro3" for local storage pool
Using "/dev/disk/by-id/scsi-SQEMU_QEMU_HARDDISK_lxd_local4" on "micro4" for local storage pool
Would you like to set up distributed storage? (yes/no) [default=yes]: yes
Unable to find disks on some systems. Continue anyway? (yes/no) [default=yes]: yes
Select from the available unpartitioned disks:
Select which disks to wipe:
Using 1 disk(s) on "micro1" for remote storage pool
Using 1 disk(s) on "micro3" for remote storage pool
Using 1 disk(s) on "micro2" for remote storage pool
Configure distributed networking? (yes/no) [default=yes]: yes
Select exactly one network interface from each cluster member:
Using "enp6s0" on "micro2" for OVN uplink
Using "enp6s0" on "micro1" for OVN uplink
Using "enp6s0" on "micro4" for OVN uplink
Using "enp6s0" on "micro3" for OVN uplink
Specify the IPv4 gateway (CIDR) on the uplink network (empty to skip IPv4): 10.166.166.1/24
Specify the first IPv4 address in the range to use with LXD: 10.166.166.100
Specify the last IPv4 address in the range to use with LXD: 10.166.166.254
Specify the IPv6 gateway (CIDR) on the uplink network (empty to skip IPv6): fd42:fc85:20e1:ccb5::1/64
Initializing a new cluster
Local MicroCloud is ready
Local LXD is ready
Local MicroOVN is ready
Local MicroCeph is ready
Awaiting cluster formation ...
Peer "micro3" has joined the cluster
Peer "micro2" has joined the cluster
Peer "micro4" has joined the cluster
Configuring cluster-wide devices ...
Error: Config key "source.wipe" may not be used as member-specific key
root@micro1:~# lxc storage list
+------+--------+-------------+---------+-------+
| NAME | DRIVER | DESCRIPTION | USED BY | STATE |
+------+--------+-------------+---------+-------+
root@micro1:~#
exit
microcloud@microcloud:~$ lxc exec micro2 -- bash
root@micro2:~# lxc storage list
If this is your first time running LXD on this machine, you should also run: lxd init
To start your first container, try: lxc launch ubuntu:22.04
Or for a virtual machine: lxc launch ubuntu:22.04 --vm
+------+--------+-------------+---------+-------+
| NAME | DRIVER | DESCRIPTION | USED BY | STATE |
masnax commented
Looks like the version of LXD you're using for each system is incompatible with MicroCloud. You'll have to run snap refresh lxd --channel latest/stable on each system, in addition to installing microovn microcloud and microceph.
nishit93-hub commented
@masnax Thanks for the solution. It worked.