Support the rapid landing of immersive experience applications

Alibaba Cloud XR platform provides computing power scheduling, 3D real-time rendering, audio and video coding and streaming, application publishing and user management and other services. Developers can host application publishing to the cloud XR platform, and users can use multiple terminals such as AR/VR/MR to join the cloud XR platform and interact with and experience 3D applications. The following is the content of his speech for everyone to read:

01 Evolution from single interaction to all-element interaction

Nowadays, immersive experience is a very popular word, and when it comes to immersive experience, it has to mention all-element interaction.

The development trend of human media is to gradually restore the real world from abstract to concrete. So, how to realize the interaction of all elements? At present, real-time interaction can be superimposed through immersion. Human beings are naturally sociable and full of curiosity, which is also the driving force of development.

The way human children perceive the world is touch, which is also a more efficient way of cognition. According to the cognitive model of human beings, more realistic simulation practice is the learning method next to Feynman learning method. The improvement of learning efficiency can undoubtedly improve production efficiency, which is also the original intention of Alibaba Cloud to produce the cloud XR platform, hoping to help the whole society and industry improve creativity and productivity.

02 Immersive experience and business technology challenges from the perspective of cloud computing

The seven-layer model in the figure below systematically divides the whole business into seven layers: experience, discovery, creator economy, spatial computing, distribution, human-machine interface, and infrastructure.

After the seven-layer model on the left is mapped to the common application fields of cloud computing on the right, new requirements for cloud computing have been generated, and these new requirements still revolve around the user experience. To sum up, Metaverse's all-element communication mode has brought unprecedented impact and challenges to cloud computing.

The challenges brought by immersion mainly come from computing, communication and business models.

Metaverse's interactive approach makes computing more complex. Take the development process from PC to mobile phone as an example. At the earliest stage, the interaction with PC can only be through the display, keyboard and mouse. Later, there was a webcam, but the interaction is still very simple, resulting in the application on PC is also very simple, the amount of data is small, and the amount of calculation is also small. Finally, many businesses can be competent using local computing capabilities, such as processing documents and other simple calculations.

The emergence of mobile phones has brought great changes, and interactive devices have become more and more. In addition to multi-touch display screens and cameras with different resolutions, they also include various sensors, such as acceleration sensors, temperature and humidity sensors, infrared sensors, distance sensors, etc. These sensors have brought more information and made the applications on mobile phones more colorful. As more and more computing needs to be processed, part of the computing of mobile applications has been transferred to the cloud, and accelerators such as GPU have also been introduced. For example, in AI and game computing scenarios, the computing power on the cloud will be called to help obtain a better user experience.

After the development of XR experience, computing scenarios become more and more abundant, such as gesture recognition, voice recognition of terminal devices, and these new interaction methods need to consume more computing power.

When the whole industry is faced with the upgrading of new interaction methods, it is not only technology that needs to be upgraded, but also business methods that need to be upgraded tenfold, mainly from the requirements of computing, communication and business ecology.

The challenge of immersive experience to computing comes from several aspects. Previous computing is more CPU computing, while immersive experience computing is more real-time rendering of 3D models, computing of AI scenes, and encoding and decoding of streaming media, all of which require very intensive computing power. The number of users of the Internet has also multiplied, requiring more computing resources to process business and a huge scale.

In communication, lower delay communication is required for real-time interaction, large-scale AI model training, and larger bandwidth communication is also required for higher resolution video transmission.

In addition, the openness of ecology needs to be further improved. Compared with other industries, the upstream and downstream enterprises involved in the immersive experience industry are more abundant and diverse, and the technology stack is also very complex. Therefore, a more open ecosystem is needed to enable various industries to cooperate with each other. Not only should all manufacturers maintain an open attitude, but also we need to help everyone build a unified standard and a unified cooperation standard to expand the industry together.

Alibaba Cloud has settled the technical problems solved in the past 10 years on the DPCA architecture. In 2017, we introduced DPCA architecture to continuously reduce IO latency and improve IO bandwidth and throughput. The fourth generation DPCA released during the 2021 Yunqi Conference can reduce the network forwarding delay to 15 us.

In addition to improving communication capabilities, we are also improving computing capabilities. In the past few years, the computing capacity of Alibaba Cloud GPU instances has increased by 100 times. In 2021, the 7-generation GPU instance was released, and there has been a huge upgrade of computing capability in AI computing, AI reasoning and immersive experience graphics computing scenarios.

In addition, we are also expanding the scale of computing. So far, the computing capacity has exceeded 1700p.

Compared with technical construction, ecological construction is more important.

03 Product capacity building to improve efficiency and ecological capacity

To prosper the industry, the most important thing is to sort out production relations. We hope to build a product system with partners and customers to make this ecosystem more perfect.

Heterogeneous computing has always been recognized as difficult to use. For this reason, we have made many optimizations in several major application scenarios through joint research and development teams. Many very efficient and easy-to-use tools are encapsulated in the graphic computing scenario, AI computing scenario and streaming media computing scenario to help users improve efficiency and improve the economy and ease of use of products.

On this basis, we also built a visual computing system, hoping to put the needs of all application scenarios of the entire industry on the cloud for promotion.

04 Cloud XR platform launched

The cloud XR platform is a real-time rendering platform for enterprise-level 3D application hosting and publishing that Alibaba Cloud cooperates with parallel cloud. In addition to supporting flat devices, it can also support VR, AR, MR and other 3D terminal devices to help users get a better user experience. Users can deploy their 3D applications on the cloud XR platform for publishing.

At present, the capability of the platform has been verified in different industries.

The cloud XR platform on the public cloud has greater advantages over private deployment.

05 Jieyun XR platform

Currently, the cloud XR platform supports GPU instances based on T4, A10 and RTX 6000, including NVIDIA vDWS license, which is very easy to use.

T4, A10 and RTX 6000 all contain key features such as RT core and Tensor core, as well as hard-coded codec units, which can help to obtain lower latency serial streams. At the same time, A10 and RTX 6000 are equipped with DPCA computing architecture, which can obtain IO with lower latency.

The cloud XR platform supports Windows server 2016 and Windows server 2019, and most mainstream 3D engines. Most terminals can be accessed through SDK or app. The XR terminal support includes Quest2, HTC VIVE, Pico Neo3, Nolo, etc. The plane can be accessed directly through terminal apps, SDKs, web pages or applets.

Whether 3D applications are finally presented in the form of VR or in the form of flat devices, the process of deploying business to the cloud is similar. Take VR as an example, import the designed model into the 3D real-time engine for editing and development through the design and modeling stage. The developed application needs to support Steam VR and open XR. After the support is completed, the application is packaged and adapted, and then released on the cloud XR platform.

The deployment process of flat 3D applications is similar, but the difference is that it does not need to support Steam VR and Open XR.

The scenario application scenarios of cloud XR are mainly distributed in two aspects, namely consumer business and enterprise business. The consumer-end business is committed to bringing more fun to end users, and the enterprise-end business is committed to helping enterprises bring higher efficiency and creativity.

Common businesses at the consumer end include VR games, e-commerce, online interactive concerts, virtual human live broadcast, and 3D product display under e-commerce scenes; The business scenarios at the enterprise end include scientific research and teaching fields, exhibitions/museums based on Metaverse, 3D conferences, virtual simulation applied in manufacturing or industry, improving efficiency through digital twinning, and service-oriented digital people.

Both enterprise-side business and consumer-side business can be easily deployed to all corners of the world through the cloud XR platform to serve users around the world.

Typical users of the cloud XR platform include Pitchen Technology, Baike Biology and the Engineering Thermophysics Institute.

"Bizhen Technology" is a leading technology company focusing on online concerts. It has held many online concerts in Alibaba Cloud in the past six months. In the latest performance, there were more than 300000 people online and more than one million people enrolled.

"Baike Biology" is the best medical digital human technology enterprise in the world at present. They have the most advanced 3D reconstruction technology of organisms in the world. At present, the medical digital human has been packaged for clinical learning research and released on the cloud XR platform.

The Institute of Thermal Engineering of Tsinghua University has realized the simulation of many large equipment through the cloud XR platform, and will open these capabilities to the public through the cloud in the future to facilitate scientific research. The above is what I share today.

Extension:

In the process of landing the cloud XR platform, Alibaba Cloud mainly provides super cloud computing power and self-developed streaming media coding technology. At the platform level, parallel cloud efficiently encapsulates rendering, coding and streaming services, integrates Alibaba Cloud's underlying IaaS capabilities, and supports the NVIDIA CloudXR platform.