GSoC contributor: Jiawei Yin
Mentors: Eugenio Perez Martin
Virtio-net device uses Control Virtqueue(CVQ) for changing device parameters. For some devices, such as vDPA, its CVQ is passed from Guest’s driver directly to Host driver, which makes it difficult for QEMU to track the status of these devices.
To solve this problem, QEMU introduces Shadow Virtqueue(SVQ) for vdpa device, which shadows the CVQ via QEMU instead of being assigned directly to Guest. During QEMU Live migration, to restore the vdpa device in the destination VM to the state in the source VM, SVQ can use CVQ’s interface and send the state as regular CVQ commands.
Due to some CVQ commands for vDPA are missing at present, this project aims to achieve support for a subset of CVQ commands, including VIRTIO_NET_CTRL_RX family commands, VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET command, VIRTIO_NET_CTRL_VLAN_ADD command, VIRTIO_NET_CTRL_MQ_HASH_CONFIG command and VIRTIO_NET_CTRL_MQ_RSS_CONFIG command for vDPA.
What's more, I also fixed some bugs and implemented performance improvements as part of this project.
| Status | Link |
|---|---|
 |
vdpa: Send all CVQ state load commands in parallel |
| Status | Link |
|---|---|
|
virtio-net: Zero max_tx_vq field for VIRTIO_NET_CTRL_MQ_HASH_CONFIG case |
- clone the submodules by
git submodule init
- install libbpf dependency by
sudo apt install libbpf-devin Ubuntu - configure the Qemu with bpf and vhost-vdpa features enabled by
/path/to/qemu/configure --enable-bpf --enable-vhost-vdpa - compile the Qemu by
make -j $(nproc)
The following table presents the state recovery implemented by the patch series, along with the options required for enabling them during Qemu startup and the corresponding bash commands to set them within the guest.
| State | VirtIO standard | options required for Qemu | commands used in guest |
|---|---|---|---|
| offload | Offloads State Configuration | ctrl_guest_offloads=on/off guest_csum=on/off guest_tso4=on/off guest_tso6=on/off guest_ecn=on/off guest_ufo=on/off |
|
| MAC Address Filtering | Setting MAC Address Filtering | ctrl_rx=on/off | ip link add ${MAC_VLAN_NAME} link eth0 address ${MAC_ADDRESS} type macvlan mode bridge |
| Packet Receive Filtering | Packet Receive Filtering | ctrl_rx=on/off ctrl_rx_extra=on/off |
ip link set eth0 promisc on/ffip link set eth0 allmulticast on/off |
| vlan | VLAN Filtering | ctrl_vlan=on/off | ip link add link eth0 name ${vlan_name} type vlan id ${vlan_id} |
| hash calculation | Hash calculation | hash=on/off | ethtool -K eth0 rxhash on/off |
| rss | Receive-side scaling | rss=on/off | ethtool -K eth0 rxhash on/off |
Use should do as the following instructions:
- boot the source and destination Qemu with the same device configuration with the wanted state mentioned above to be supported by
/path/to/qemu \
-netdev -netdev type=vhost-vdpa,id=${name},vhostdev=/path/to/vdpa-device,x-svq=true \
-device virtio-net-pci,netdev=${name},mq=on,ctrl_vq=on,guest_tso4=on,guest_tso6=on,guest_ecn=on,guest_ufo=on,guest_announce=off,${options-required-for-Qemu} \
...- config the state in guest in source by execute
${commands-used-in-guest} - execute the live-migration in source destination, the device state will be restored in the destination Qemu
I am incredibly thankful to my mentor, Eugenio, whose unwavering support helped me a lot during this project. Throughout my internship, his guidance has been invaluable, providing clarity to my inquiries and deepening my understanding through insightful discussions on pertinent subjects. Thanks to Eugenio's mentorship, I was able to quickly get up to speed on this project and ultimatly complete it successfully.
Furthermore, I would like to extend my heartfelt appreciation to the diligent QEMU community developers, whose constructive feedback greatly enhanced the quality of my patches. Their generous contributions have significantly contributed to the refinement of my work.
Over the course of this internship, I had the opportunity to learn the knowledge about the vDPA framework and gained profound insights into the process of state changing within QEMU's vhost-vdpa backend. It brings me immense joy to successfully implement the state restore feature for QEMU's vhost-vdpa device. This experience has been exceptionally enriching and has undoubtedly broadened my horizons!