Snapshots for lvm-vdo
Opened this issue · 7 comments
Hi there! Wanted to ask if LVM snapshots are supported for lvm-vdo? From the RedHat documentation, it's not supported to create LVM snapshots below VDO: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/vdo-qs-requirements but as far as I understand, it's referring to VDO when it was placed on top of a VDO volume. Now, we create LVM of VDO type. So the question basically is if it's supported to create snapshots of lvm-vdo volume or there are some issues/limitations?
lvm-vdo does not currently support snapshots itself. Our recommendation is to create an lvm-thin target on top of the VDO target, and use the thin target to create and manage snapshots.
We are doing some investigation on ways to simplify this scenario, but I don't currently have a timeline for when those may be available.
Got you. One more question. I've managed to create a snapshot of lvm-vdo volume using the following command example: lvcreate --size 100M --name snap --snapshot /dev/vg001/origin (of course, pointed to lvm-vdo volume) and a snapshot was created successfully. Do I understand correctly that even though the snapshot itself can be created, it's still not guaranteed to work properly with lvm-vdo and hence, not supported?
As long at the dm-thin target is on top of the vdo volume, your configuration is fine and is supported.
I was trying to say the the vdo itself does not know about the snapshots in this configuration, so the snapshots have to be managed through the dm-thin target instead. But vdo is able to deal with as many snapshots as you have space to create.
Got you. Thanks for clarifying this. One more question, I read that loopback devices are not supported below VDO. Could you please tell if VDO can be placed on top of an NBD device and if it's supported?
I don't think we've tested VDO on top of nbd specifically, so I can't really say it's supported. However I don't really see a reason why it would present a problem.
The problem with loopback devices, as I recall, is that we don't want the devices below VDO to need to allocate local memory when writing. It is possible to get into a situation where something wants to flush the Linux page cache, so it sends some writes through VDO, which triggers a memory allocation on the underlying device. But if memory is tight, the only way to reclaim memory may be to flush the page cache... So the machine can end up in a livelocked state where it can't complete any writes or reclaim any memory. This problem isn't actually unique to VDO devices, or particularly common, but it's bad when it happens so we advise against using device stacks that can have this issue.
Got you. Regarding the snapshots, this makes me wonder, since lvm-vdo snapshots are not supported and the best way to make snapshots is with lvm-thin target on top of the VDO, is it better to avoid any snapshots below VDO? For example, if I have a VM with lvm-vdo for deduplication, is it better to avoid doing snapshots of that VM (on a hypervisor level) as well? And if it’s better to avoid doing snapshots below, is there a specific reason for this, like may it cause consistency issues?
Using snapshots on the storage underneath VDO is not recommended. VDO does a lot of metadata management in RAM, so a snapshot of the underlying storage device of an active VDO volume won't capture all the necessary state that one would like it to. And if that snapshot is subsequently restored while the VDO volume is active, it can lead to metadata corruption and lost data because there is no way to inform the active processes that the underlying data has changed outside of its control. (If you shut down the VDO device before restoring snapshots, it would probably work okay, but that seems like it would be inconvenient to manage. Even in that case the snapshot might not include all the data you expect.)