By SIG for High-Performance Network
Today's cloud computing is in full swing. We often see the word virtio on cloud computing platforms, which appears on various devices of cloud computers.
What role does it play in the virtualization technology field?
Virtio is an I/O semi-virtualization solution, a set of general-purpose I/O device virtualization programs, and an abstraction of a set of general-purpose I/O devices in the semi-virtualization Hypervisor.
Generally speaking, virtio consists of three parts:
The first two parts are the driver and device implemented by virtio in different operating systems and computer simulators. There are no strict regulations on how to implement the driver and device, which are different based on different environments. However, the interaction between driver and device must strictly abide by the definition of virtio spec. Then, it can be guaranteed that all OS functions properly in simulators on different hosts.
Virtio Spec is the Essence of Virtio.
Virto spec defines a set of virtqueue mechanisms to enable communication between guest and host. This mechanism can be applied to various devices (network device, block device, etc.). Therefore, virtio spec developed a variety of devices that mostly covered virtualization scenarios.
Virtio is a semi-virtualized technology. If the device is fully virtualized, the hypervisor must intercept all hardware request instructions, which has a huge impact on performance. The guest is essentially a process on a host, which is simple to interact with a hypervisor. It can interact based on the communication mode between processes. This requires the guest to cooperate. Virtio implementation based on it can achieve higher performance. That is why virtio is widely supported.
The very first specification Rusty Russell drafted in 2012 was informal. After that, the Virtio Technical Committee was responsible for virtio spec and released several versions. Under the auspices of the committee and with the extensive participation of the community, new features and capabilities have been added to virtio spec. Virtio Spec has been updated to v1.2.
After more than half a year's efforts, Virtio 1.2 was officially released.
First, Virtio Spec 1.2 is fully compatible with earlier versions. All features defined by virtio spec are negotiated through features, so the implementation can be smoothly transitioned. There is no incompatibility between different versions after the upgrade. In terms of devices, Virtio Spec 1.2 supports some new devices: virtio-pmem, virtio-fs, and virtio-rpmb. Some mainstream devices also support new features. For example, virtio-net added these new features:
In addition, virtio core added some new basic features:
Overall, the release of Virtio Spec 1.2 has brought many exciting new features. I believe that after these functions are officially implemented, we will get a more modernized virtio. This is also an opportunity to drive manufacturers to improve their virtio implementation.
The Per-virtqueue reset in Virtio Spec 1.2 was drafted by Xuan Zhuo of Alibaba Cloud. Introducing this feature solves the problem that virtio-net does not support queue-level reset operations. This is a common function in many modernized network interface controllers and is the basic capability to realize many functions. It is necessary to introduce this feature to make virtio-net support more capabilities. However, Per-virtqueue reset is not limited to the virtio-net device, which is the basic capability of virtio. I believe other virtio devices will support this feature gradually.
The per-virtqueue reset is initiated by the driver for a queue and notifies the device based on transport (such as PCIe). The device stops using the queue, and the driver can re-enable the queue after resetting. Virtio spec defines the detailed interaction process and information in this procedure. The following is the detailed process defined in virtio spec:
The reset operation of the driver for the queue has been completed.
This is a complete reset and re-enable procedure. Theoretically, re-enable is optional.
Resetting queues is a common function for many modernized hardware devices, so its introduction makes virtio devices more modernized. The emergence of virtio in the early days came with the demand for high performance. We paid more attention to its basic functions in performance and ignored some advanced functions. Per-virtqueue reset makes virtio flexible in the use of queues. For example, we can implement the following two functions based on Per-vertqueue reset:
Per-virtqueue reset, we can implement the adjustment to the ring size of the network interface controller queue. Currently, the common network interface controller supports the use of ethtool -G eth0 rx <new size> tx <new size> to adjust the size of the queue, but the original virtio-net did not support such a simple function. Now, based on the Per-virtqueue reset, we will be able to use this command under Linux to adjust the size of the queue soon.Per-virtqueue reset can be regarded as a rapid resource recovery mechanism. For example, in the case of virtio-net, we must first wait for new data packets or hardware to complete the transmission of data packets. Then, we can complete the recycling of buffer resources. Now, based on Per-virtqueue reset, the driver can call reset to quickly release the buffer resources occupied by a device on a queue instead of waiting passively, thus realizing rapid resource recycling. This allows virtio-net to support advanced functions (such as AF_XDP) and achieve high-performance packet forwarding under the Linux kernel framework.More application scenarios are waiting for you to explore.
The preceding contents are the definition of virtio spec. As the drafter of this feature, Xuan Zhuo moved to open-source-related implementations in the Linux kernel community after this feature officially entered virtio spec. Other device backend implementations are also in progress.
per-virtqueue reset function in the virtio core should be able to enter the Linux kernel mainline soon. This patch set contains a virtio-net implementation that supports ring size.qemu/vhost-user(DPDK)/vhost-kernel is in progress.You will be able to experience this function on the new version of Qemu and Linux kernel soon.
As a large-scale cloud computing technology, Virtio is a basic technology used by major domestic cloud manufacturers. However, we have received little feedback and had rare participation over the years. Recently, we noticed that Alibaba and other domestic cloud manufacturers began to actively participate in it, including fixing bugs, setting standards, and promoting implementation. It is a good thing for the long-term development of domestic cloud technology. We hope there will be more participation to jointly promote the development of virtio.
We set up a virtio interest group under the high-performance network SIG in the community to support more features and optimize the landing of the OpenAnolis operating system better. We can enhance mutual communication and understanding in this area by strengthening the cooperation between various virtio-related teams and companies in the community, thus promoting virtio and its better development in OpenAnolis. I hope that more community partners can participate in the future. You are welcome to participate in the discussion and ask questions or propose needs. We can build the virtio SIG in the community together equally. This is a win-win.
Win-win, Equality, and Open-Source
Ding Xuefeng (Alibaba), Heng Qi (Alibaba), Xu Kangjie (Alibaba), Ma Jiangying (Intel CNDP), Gu Zhengguo (Jaguar Microsystems), Zhang Mingli (Jaguar Microsystems), Yang Tao (Jaguar Microsystems), Sun Chuanming (Jaguar Microsystems), Gong Youhua (YunSilicon), Meng Xianghong (YunSilicon), Chen Yang (YunSilicon), and Qian Rong (YunSilicon)
You are welcome to join the Virtio SIG High-Performance Network.
SIG Address Link: https://openanolis.cn/sig/high-perf-network
83 posts | 5 followers
FollowOpenAnolis - September 6, 2022
AlibabaCloud_Network - September 11, 2018
muyuan - January 11, 2021
OpenAnolis - December 7, 2022
OpenAnolis - February 27, 2023
Alibaba Clouder - March 19, 2020
83 posts | 5 followers
Follow
Elastic High Performance Computing Solution
High Performance Computing (HPC) and AI technology helps scientific research institutions to perform viral gene sequencing, conduct new drug research and development, and shorten the research and development cycle.
Learn More
Elastic High Performance Computing
A HPCaaS cloud platform providing an all-in-one high-performance public computing service
Learn More
Remote Rendering Solution
Connect your on-premises render farm to the cloud with Alibaba Cloud Elastic High Performance Computing (E-HPC) power and continue business success in a post-pandemic world
Learn More
Function Compute
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.
Learn MoreMore Posts by OpenAnolis