Manual update dependencies in rust-vmm crates - and provide releases
andreeaflorescu opened this issue · 12 comments
This update is needed because otherwise there will be crates using vmm-sys-util 0.10.0 and others using 0.11.0. The reason behind this is the (wrong) way in which we specify dependencies in a few of the rust-vmm crates. Specifically, some crates will just pick up the latest available version of a crate even if it introduces breaking changes (i.e. picking up 0.11.0 because the dep is specified as >= 0.9.0). We're progressively fixing this by always having dependencies specified with caret, but this work was not yet completed for a few crates. As such, we need to:
- fix the way we specify dependencies for the crates (use caret instead of >=)
- provide a new release
The order in which we provide releases matters because of the dependency between crates.
For example, we first need to update kvm-bindings, publish it, and then update kvm-ioctls by specifying a new version for both vmm-sys-util and kvm-bindings.
This issue is tracking this one time effort. We're not tracking here a saner way to do the updates, as this is going to happen afterwards. Tracking issues: #135 & TBD (for releases).
I went through all of the rust-vmm crates and created the following table that keeps track of all their dependencies and whether they currently rely on (a) non-caret requirements and (b) outdated versions
| Crate | Version | Uses caret deps? | rust-vmm deps at fixed versions (outdated are bold) | rust-vmm deps at non-caret versions |
|---|---|---|---|---|
| kvm-ioctl | 0.11.0 | yes | vmm-sys-util@0.10.0 kvm-bindings@0.5.0 (MERGED) | |
| vmm-sys-util | 0.11.0 | yes | ||
| kvm-bindings | 0.5.0 | yes | vmm-sys-util@0.10.0 (MERGED) | |
| event-manager | 0.2.1 | no | vmm-sys-util>=0.8.0 | |
| vfio-ioctls | 0.1.0 | no | kvm-bindings~0, kvm-ioctls~0, vm-memory>=0.6, vfio-bindings~0, vmm-sys-util>=0.8.0 (NO PR OPEN) | |
| vfio-bindings | 0.3.1 | no | vmm-sys-util>=0.8.0 (NO PR OPEN) | |
| vhost-device-gpio | 0.1.0 | no | virtio-queue@0.6. vhost-user-backend@0.7.0 | vmm-sys-util=0.10.0. vm-memory>=0.8, virtio-binding>=0.1 (NO PR OPEN) |
| vhost-device-i2c | 0.1.0 | no | vhost-user-backend@0.7.0, vhost@0.5, virtio-queue@0.6 | vm-memory>=0.8, vmm-sys-util=0.10.0 (NO PR OPEN) |
| vhost-device-rng | 0.1.0 | no | vhost@0.5, vhost-user-backend@0.7.0, virtio-queue@0.6 | vm-memory>=0.8, vmm-sys-util=0.10.0, virtio-bindings>=0.1 (NO PR OPEN) |
| vhost-device-vsock | 0.1.0 | no | vhost@0.5, vhost-user-backend@0.7.0*,* virtio-queue@0.6*, viortio-vsock@0.1.0* | vm-memory>=0.8, vmm-sys-util=0.10.0, virtio-bindings>=0.1 (NO PR OPEN) |
| mshv-ioctls | 0.1.1 | no | vmm-sys-util>=0.8.0 (NO PR) | |
| mshv-bindings | 0.1.1 | no | vmm-sys-util>=0.2.1 (NO PR) | |
| virtio-blk | 0.1.0 | no | vmm-sys-util@0.10.0, vm-memory@0.9.0, virtio-device, virtio-bindings(MERGED) | |
| virtio-console | 0.1.0 | no | virtio-queue@0.6.0, virtio-bindings@0.1.0, vm-memory@0.9.0(MERGED) | |
| virtio-vsock | 0.1.0 | no | virtio-queue@0.6.0, virtio-bindings@0.1.0, vm-memory@0.9.0(MERGED) | |
| virtio-queue | 0.6.1 | no | vm-memory@0.9.0, vmm-sys-util@0.10.0, virtio-bindings@0.1.0(MERGED) | |
| virtio-queue-ser | 0.3.0 | no | virtio-memory@0.9.0, virtio-queue@0.6.0(MERGED) | |
| virtio-device | 0.1.0 | no | vm-memory@0.9.0, virtio-queue@0.6.0, virtio-bindings (MERGED) | |
| virtio-bindings | 0.1.0 | no | ||
| vm-fdt | 0.2.0 | yes | ||
| vm-device | 0.1.0 | yes | ||
| vm-allocator | 0.1.0 | no | ||
| vm-memory | 0.9.0 | yes | vmm-sys-util@0.10.0 (MERGED) | |
| vhost | 0.5.0 | yes | vmm-sys-util@0.10.0 (No PR), vm-memory@0.9.0 | |
| vhost-user-backend | 0.7.0 | yes | vmm-sys-util@0.10.0 (No PR), vm-memory@0.9.0, vhost@0.5, virtio-queue@0.6, virtio-bindings@0.1 | |
| vm-superio | 0.6.0 | no | vmm-sys-util>=0.7.0 (MERGED) | |
| vm-superio-ser | 0.2.0 | no | vm-superio@=0.6.0 | vmm-sys-util>=0.7.0 (MERGED) |
| linux-loader | 0.7.0 | yes | vm-memory@0.9.0 (MERGED) |
Going through the dependency graph, updates need to be done in the following order (all crates on the same level can be updated in parallel):
- vm-superio, vm-memory and kvm-bindings, event-manager
- vhost, vm-superio-ser, virtio-queue, virtio-blk, kvm-ioctl
- vm-vhost-user-backend, the various
vhost-*,vfio-*andmshv-*crates
Based on @roypat's analysis, we need to go through 3 stages (corresponding to the number list above).
Stage 1
vm-superio - DONE
Changes Needed:
- move to caret dependency - @roypat
- release - @roypat
- publish release - @andreeaflorescu
Blocks: vm-superio-ser
event-manager - DONE
Changes Needed:
- move to caret dependency - @chao-p DONE
- update rust-vmm-ci: rust-vmm/event-manager#97 @aghecenco DONE
- release - @aghecenco DONE
- publish release - @aghecenco DONE
Blocks: N/A
vm-memory - DONE
Changes Needed:
- update vmm-sys-util - @andreeaflorescu - rust-vmm/vm-memory#209
- update rust-vmm-ci - @andreeaflorescu - DONE
- release - @andreeaflorescu - rust-vmm/vm-memory#209
- publish release - @andreeaflorescu
Blocks:
- vhost* crates
- vfio-* crates
- virtio-* crates
kvm-bindings - DONE
Changes Needed:
- release - @roypat
- publish release - @aghecenco
At this point, all stage 1 crates have PRs open fixing this issues. Next step is their crates.io release, so we can start working on the Step 2 crates.
kvm-bindings v0.7.0 is released ✅
event-manager CI is repeatedly failing with a strange permissions issue on a local dir (seemingly unrelated to these releases, more likely a Buildkite thing and one more problem to fix)
kvm-bindings v0.7.0 is released ✅ event-manager CI is repeatedly failing with a strange permissions issue on a local dir (seemingly unrelated to these releases, more likely a Buildkite thing and one more problem to fix)
Is that v0.6.0 or a newer version of kvm-bindings is planned to release?
kvm-bindings v0.7.0 is released ✅ event-manager CI is repeatedly failing with a strange permissions issue on a local dir (seemingly unrelated to these releases, more likely a Buildkite thing and one more problem to fix)
Is that v0.6.0 or a newer version of kvm-bindings is planned to release?
It is 0.6.0.
Stage 2
vm-superio-ser - DONE
kvm-ioctls - DONE
- update vmm-sys-util @roypat - DONE
- release - @roypat - DONE
- publish release @andreeaflorescu / @aghecenco
virtio-queue - DONE
- update dependencies (vmm-sys-util) - @roypat
- release - @roypat
- publish release - @andreeaflorescu
Block: virtio-vsock
linux-loader - DONE
I know that a release is missing for (at least) vhost-user-backend.
I know that a release is missing for (at least) vhost-user-backend.
This is all done now. Thanks to @stefano-garzarella. vfio-ioctls & vfio-bindings remain along with some of the virtio backend devices.
I know that a release is missing for (at least) vhost-user-backend.
This is all done now. Thanks to @stefano-garzarella.
vfio-ioctls&vfio-bindingsremain along with some of the virtio backend devices.
@jiangliu @liuw @sameo @sboeuf can one of you take care of the two crates as well? What is important is properly setting the crate requirements to caret (if they have any that are not properly specified), and provide a release.