Queen SDK is a smart retouching SDK developed by the Alibaba Cloud video cloud team. This SDK provides a wide range of features including image retouching, face retouching, chroma key, stickers, makeup, gesture recognition, body shaping, hairdressing, and augmented reality (AR) writing. You can use these features in live streaming, video conferencing, and short video production scenarios. This topic describes the benefits, core features, and scenarios of Queen SDK.
Introduction
Alibaba Cloud Queen SDK is an easy-to-use smart retouching SDK that provides a variety of powerful features. Queen SDK adopts the in-house intelligent vision algorithm, and face and human body detection and recognition technologies. It provides video creators with editing and processing capabilities such as basic face retouching, advanced face retouching, face shaping, makeup, stickers, chroma key, and gesture and posture recognition on mobile devices. Queen SDK provides sufficient capabilities to meet the diverse retouching requirements during live streaming and video production. You can use it on Android, iOS, and web.
Benefits
The proprietary facial keypoint detection technology recognizes 106 basic points and 280 high-accuracy points. This technology makes retouching effects more vivid.
The intelligent vision algorithm and real-time rendering technology are optimized regularly to continuously improve user experience.
Face retouching and shaping effects, filters, stickers, and materials are continuously updated and improved to provide more options for retouching.
Comprehensive developer support provides timely solutions to issues, and ensures the reliability of services.
Features
Feature | Description |
Face retouching | Supports whole face and manual face retouching. Whole face retouching provides five styles. Manual face retouching lets you whiten, smoothen, and brighten the skin, add rosy cheeks, sharpen images, adjust the saturation and contrast, remove eye bags, nasolabial folds, wrinkles including neck and forehead wrinkles, whiten teeth, brighten eyes, and apply lipstick and blush. You can adjust the intensity level for each effect. |
Face shaping | Supports whole face and manual face shaping. Whole face shaping provides five styles. Manual face shaping supports the following effects:
|
Makeup | Supports whole face makeup and manual makeup. Whole face makeup styles: simple, pure, vintage, and mature. Manual makeup effects: lipstick, aegyo-sal, blush, eyebrow, highlight, eyeshadow, eyeliner, and eyelash. You can adjust the intensity level for each effect. Makeup effects adapt to facial movements and expressions in real time. |
Face play | Provides funny effects such as pixelation and facemasks based on face detection and tracking technologies. |
Filters | Provides 34 filters, with more being added. Real-time rendering is supported. |
Stickers | Provides 79 stickers, including static and animated ones, with new ones being added. Stickers seamlessly match up with faces and adapt to facial movements and expressions in real time. |
Chroma keying | Supports both blue and green screens. The feature precisely identifies and separates the human body from complex backgrounds under various lighting conditions and angles, while maintaining real-time body tracking. |
Image matting | Supports background replacement in real-world environments. The feature precisely identifies and separates the human body from complex backgrounds under various lighting conditions and angles, while maintaining real-time body tracking. Note Background blur, transparent backgrounds, and background replacement can be implemented using this feature. |
Gesture recognition | Accurately detects and captures key points of hands and gestures, and recognizes 25 popular gestures such as numbers, heart, fist, claw, and Vulcan salute, and a variety of dynamic gestures. |
Movement detection | Identifies 13 static postures such as hand raising, hand heart, arms akimbo, and superman pose, and 9 movements including rope jumping, jumping jack, squat, arm circle, diamond stretch, chest stretch, sit-up, push-up, and kneeling push-up. The number of times these movements are done is counted in real time. |
Body shaping | Lets you slim bodies, legs, arms, necks, and waists, lengthen legs, resize heads, and enlarge breasts. The natural body shaping effects make bodies look well-proportioned. |
Hairdressing | Accurately identifies hair areas in various postures and background environments, and provides 8 hair colors. |
AR writing | Tracks hand and finger movements, so that you can write or draw on the screen in real time. This feature is useful in live streaming, teaching, and advanced interaction. |
Animoji | Captures different angles and expression changes of human faces. Animojis can be set to change in certain ways based on the facial expression recognition algorithm. This creates a fun and entertaining effect. It supports 51 different expressions, including winking, blinking, and mouth, eye, and eyebrow movements. |
Liveness detection | Determines whether the person is a live person by analyzing movements such as mouth opening, head shake to the right, and lifting or lowering the head. |
Unusual behavior detection | Identifies whether the person is holding a mobile phone or wearing earphones, and whether there is a display monitor. It also detects unusual behaviors such as leaving the camera range, standing up, sitting down, and hand raising. |
Face authentication and similarity check | Authenticates faces by grading the similarity from various angles based on the facial keypoint detection and characteristics calculation algorithms. |
Common scenarios
Live streaming
Applies to various live streaming scenarios that require real-time retouching.
Long and short videos
Provides a rich set of effects for video production.
Online education
Helps teachers improve their appearance in the class. The natural retouching effects make teachers look more approachable.
Video conferencing
Provides real-time retouching and interaction effects during video conferences.