All Products
Search
Document Center

ApsaraVideo Live:Overview

Last Updated:Mar 01, 2024

Queen SDK is a smart retouching SDK developed by the video cloud team of Alibaba. This SDK provides a wide range of features including image retouching, face retouching, chroma key, stickers, makeup, gesture recognition, body shaping, hairdressing, and augmented reality (AR) writing. You can use these features in live streaming, video conferencing, and short video production scenarios. This topic describes the benefits, core features, and scenarios of Queen SDK.

Introduction

Alibaba Cloud Queen SDK is an easy-to-use smart retouching SDK that provides a variety of powerful features. Queen SDK adopts the in-house intelligent vision algorithm and face and human body detection and recognition technologies. It provides video creators with editing and processing capabilities such as basic face retouching, advanced face retouching, face shaping, makeup, stickers, chroma key, and gesture and posture recognition on mobile devices and PCs. Queen SDK provides sufficient capabilities to meet the diverse retouching requirements during live streaming and video production. You can use Queen SDK on Android, iOS, Windows, macOS, and Web.

Benefits

  • The proprietary facial keypoint detection technology can recognize 106 basic points and 280 high-accuracy points. This technology makes retouching effects more vivid.

  • The intelligent vision algorithm and real-time rendering technology are optimized on a regular basis to continuously improve user experience.

  • Face retouching and shaping effects, filters, stickers, and materials are continuously updated and improved to provide more options for retouching.

  • Comprehensive developer support provides timely solutions to issues, and ensures the reliability of services.

Features

Feature

Description

Face retouching

Supports auto face retouching and manual face retouching. Auto face retouching provides five intensity levels. Manual face retouching allows you to whiten and smoothen the skin, add rosy cheeks, sharpen images, adjust saturation and contrast, remove eye bags, nasolabial folds, wrinkles, cervical stripes, and forehead wrinkles, whiten teeth, brighten eyes, apply lipstick and blush, and brighten the skin. You can adjust the intensity level for each effect.

Face shaping

Supports auto face shaping and manual face shaping that includes 18 facial adjustments. Auto face shaping provides five types of styles. Manual face shaping supports the following effects:

  • One-way shaping: You can slim faces and lower jaws, narrow cheekbones, enlarge and widen eyes, and make smiles.

  • Two-way shaping: You can adjust the length of faces, chins, and philtrums, the width of chins, the thickness of lips, the distance between eyes, and the position of hairlines. You can also shape lips and canthi, and resize noses and nose wings.

Makeup

Supports auto makeup and manual makeup. Auto makeup provides four styles, including simple, pure, vintage, and mature. Manual makeup provides eight effects, including lipstick, aegyo-sal, blush, eyebrow, highlight, eyeshadow, eyeliner, and eyelash. You can adjust the intensity level for each effect. Makeup effects follow faces based on facial movements and expressions.

Face play

Provides funny effects such as pixelation and facemasks based on face detection and tracking technologies.

Filters

Provides 34 filters. More filters will be provided in the future. Real-time rendering is supported for a better user experience.

Stickers

Provides 79 stickers, including static and animated stickers. More stickers will be provided in the future. Stickers well fit faces and follow faces based on expressions and movements in real time.

Chroma key

Supports both the blue screen and green screen for chroma key. Human bodies can be recognized and extracted from the background from all angles, even if the background is busy and the human bodies are moving.

Background replacement

Supports real-life background replacement. Human bodies can be recognized and extracted from the background from all angles, even if the background is busy and the human bodies are moving.

Gesture recognition

Accurately detects and captures key points of hands and gesture movements, and recognizes 25 mainstream gestures such as number, heart, fist, claw, and Vulcan salute and a variety of dynamic gestures.

Movement detection

Identifies 13 static postures such as raising hands, hand heart, arms akimbo, and superman pose, and 9 movements including rope jumping, jumping jack, squat, arm circle, diamond stretch, chest stretch, sit-up, push-up, and kneeling push-up. These postures and movements are counted in real time.

Auto face retouching

Supports intelligent adjustment of retouching parameters in different background environments, and calculates the most suitable retouching parameters in real time based on objective conditions such as the background, light, and skin color.

Body shaping

Allows you to slim bodies, legs, arms, necks, and waists, lengthen legs, resize heads, and enlarge breasts. The natural body shaping effects make bodies well-proportioned.

Hairdressing

Accurately identifies hair areas in various postures and background environments, and provides eight hair colors.

AR writing

Tracks hand and finger movements to allow you to write and draw in the air in real time. This feature can be used in a variety of business scenarios such as live streaming, teaching, and advanced interaction. Examples of advanced interaction include lucky draw, red envelope, and quiz.

Animojis

Captures different angles and expression changes of human faces and drives specific Animojis to make corresponding changes based on the proprietary face recognition and expression recognition algorithms. This creates an entertaining and funny effect. Fifty-one expressions such as single-eye wink, double-eye wink, mouth movements, eye movements, and eyebrow movements are supported.

Liveness detection

Determines whether the person is a live person by analyzing movements such as mouth opening, head shake to the right, and lifting or lowering the head based on the proprietary face recognition algorithm.

Unusual behavior detection

Identifies whether the person is holding a mobile phone or wearing earbuds or a headphone and whether there is a display monitor, and detects unusual behaviors such as leaving, standing up, sitting down, and raising hands based on the proprietary object detection and behavior detection algorithms.

Face authentication and similarity check

Authenticates faces by grading the similarity from various angles based on the facial keypoint detection and characteristics calculation algorithms.

Common scenarios

  • Live streaming

    Applies to various live streaming scenarios that require real-time retouching.

  • Long and short videos

    Provides a rich set of effects for video production to meet your business requirements.

  • Online education

    Helps teachers improve their appearance in the class. The natural retouching effects make teachers look more approachable.

  • Video conferencing

    Provides real-time retouching and interaction effects during video conferences.