The purpose of augmented reality is to combine the physical and virtual worlds.

As the cornerstone of the next virtual world, XR gadgets have suddenly caught the attention of big technology corporations as the metaverse becomes increasingly popular. As a technology business specializing in human-computer interface technology and AI hardware R&D, Rokid has spent a significant amount of time developing AR glasses and has gathered a wealth of technological expertise.

Rokid's product technology director, Zhao Weiqi, will share how Rokid may boost AR glasses with AI and 5G to create a 3D space and enhance immersion.

The purpose of augmented reality is to combine the real and virtual worlds. Rokid has been working on using AR technology to create a more natural interface between reality and virtuality.

To achieve this goal, we may need the following skills:

Perception: Using the sensors and modules on the hardware and the numerous application functionalities on the software, users may access more information about themselves and their surroundings.

Comprehension: With the aid of AI, information is acquired based on perception, including people, things, the environment, behavioral intents, and peripheral data.

Cooperation: Humans cannot avoid sociability and collaboration, and we must communicate and collaborate frequently. How to create a better, less expensive, and more effective collaboration is crucial in the present context.

Presentation: Richer interactions might be feasible if other types of multimodal material display, such as 2D audio/video and 3D models, were used.

Content creation and digital asset management: From 2D to 3D, augmented reality imposes a multitude of content production and digital asset management requirements. In this context, the capacity to develop and manage digital materials efficiently is essential.

Rokid is prepared to incorporate the following software and hardware capabilities:

Chips: Rokid and Hisilicon have collaborated to build the XR platform with chip-level capabilities.

Solutions: Rokid has very mature solutions in various industries: B2C, B2B2C, B2B, and B2G.

Voice interaction: Spending several years, Rokid has been developing voice interaction technology and has achieved 99 percent accuracy in offline voice recognition.

Hardware: The design of the hardware is Rokid's forte. Rokid integrates ID, ME, and the supply chain to deliver superior hardware capabilities so that gadgets have increased comfort and enhanced fundamental capabilities.

Optical display: Rokid offers the most advanced optical waveguide technology globally, which has been implemented in a vast array of devices.

Visual algorithms: Augmented reality glasses employ visual algorithms to interpret, observe, and grasp the surrounding world to provide relevant material.

Therefore, Rokid can now incorporate the capabilities of AR and AI more effectively.

In terms of specific details, the Rokid software layer delivers outstanding multimodal interaction capabilities, including an operating system compatible with 2D and 3D ecosystems, a feature-rich SDK, easy development tools and a developer community and application publication platform.

Both partners and developers will be able to implement these features rapidly. Moreover, Rokid insists on developing futuristic AI and AR-based technical capabilities in the whole XR system. This encompasses the mainstream chip platform, the AR hardware platform, the AR headband/headset, the monocular/binocular glasses, the all-in-one/split device design, the consumer/industrial grade device configuration and design, etc.

Rokid features a more comprehensive perception and understanding mechanism than the standard operating system. Current perception system engines include platform support, AI services, augmented reality services, and cloud storage. Rokid also offers a multitude of platform SDKs and essential SDKs for operating systems and industries.

Current Rokid partners include application developers, application designers, solution providers, and system integrators. Rokid can equip them with hardware and software capabilities and has developed the framework of the XR metaverse to assist individuals in releasing their inner creativity.

Rokid's products also feature several technical advancements.

The recently announced Rokid Air, a B2C AR device, features a lightweight and portable design, a wide viewing angle, excellent resolution, and two high-quality speakers that provide surround sound. In addition, Rokid Air offers a robust voice interaction feature that allows users to do a considerable number of tasks with relatively simple voice commands, entirely freeing their hands and letting them experience AR in an immersive manner.

In addition, Rokid Air allows for adjusting myopia below 500 degrees, allowing myopic people to use this device through the knob effortlessly. The plug-and-play capability, which is compatible with cell phones and a variety of entertainment devices, also provides increased flexibility and convenience.

Current uses of Rockid Air include mostly the following:

Audio and video experience: Rokid Air can be stored at any time, whether at home, on an aircraft, or on a train, and provides an audio and video experience. You may link your mobile phone and other entertainment devices with Rokid Air. An exclusive private edition of the huge screen theater will be shown to you on a 120-inch high-definition screen.

Games: The Rokid app store provides a vast selection of game materials that create 3D settings in the real world to enhance the gaming experience. Rokid Air offers mobile gaming, including connections to game consoles and gamepads. Besides mobile, Switch, SteamDeck, and even PS, Rokid Air can "take it all in" and deliver a massive screen with minimal latency and good picture quality to its customers.

Rokid Air Pro is a B2B2C augmented reality device; its primary distinction from the Rokid Air is its interactivity. First, the Rokid Air Pro features a directed sound design so that only the user can hear it, and it does not disturb those nearby. Additionally, Rokid Air Pro offers additional interactive features. In addition to voice interaction, the camera in the Rokid Air Pro provides enhanced image recognition capabilities. Through the camera, Rokid Air Pro can detect more scenes and feel its surroundings to show and give additional scenario-based apps and interactions.

Rokid Air Pro has been implemented in a variety of B2B2C settings, including large museums and shows, where it may provide a novel presentation method in which the user's curiosity and thirst for information are completely stimulated. Rokid Air Pro allows consumers to have more knowledge dissemination and experience outside of the exhibition venue.

Using augmented reality devices, Rokid's worldwide guided tour initiative with the Suzhou Museum and the British Museum enables users to swiftly and easily access the exhibits' explanations. This is tough to do with the standard guides and docents. In addition to speech introduction, AR devices can combine video, images, and 3D models to explain difficult-to-understand aspects of foreign culture in greater detail, allowing users to get the information they need more quickly and enhance the quality of their stay.

Rokid collaborated with Tencent and the world heritage—Forbidden City —on a special digital display that allowed people to see the Forbidden City's famed summer refuge, the Juan Qin Zhai, using AR devices. Users may circumnavigate the entire virtual world and discover the historical palace's previously difficult-to-observe shapes, materials, colors, and other architectural features. This can improve the inheritance and spread of the culture of the Forbidden City to some extent.

Another project in which Rokid cooperates with the Forbidden City is the Forbidden City Painting and Calligraphy Exhibition.  There is always a stumbling block in the process of viewing paintings and calligraphy, namely that viewers can only see the surface and are unaware of the tale or situations depicted. Using augmented reality, we can transform 2D into 3D, allowing viewers to see the actual shape of paintings and calligraphy and reveal the tales and origins behind them, particularly the culture, in a more efficient manner.

Besides, Rokid's augmented reality viewing platform, collaborating with Tencent Games, reproduces the game scenario in League of Legends, allowing viewers to see a dragon flying above the Oriental Pearl Tower while wearing AR glasses. In addition, augmented reality may merge numerous actual and virtual components of a game, such as the game site, close-ups of the game participants, and key indications, which is now one of the most significant paths of AR development.

In the future, Rokid intends to expand its exploration of the metaverse and merge AR with AI through software/hardware to produce more practical, intelligent, and natural content presentation and interaction solutions. By enabling the scene, the scene's users, and the scene's partners and developers, Rokid will help users obtain more diversified and multimodal experiences inside the metaverse ecosystem.