The short video SDK provides the duet recording feature that allows you to record a duet that consists of a sample video and a video that is captured by a camera. The two videos are arranged in the specified layout, such as left-right split-screen, up-down split-screen, or picture-in-picture (PiP). Each frame of the duet contains images from the two videos, and the audio of the sample video is used as the audio of the duet. The duet recording feature is an upgraded version of the basic recording feature, which adds a new local video track.

Supported editions

Edition Supported
Professional Yes
Standard Yes
Basic No

Terms

This section describes the terms that help you better understand the duet recording feature. For more information, see Duet recording, Track, and Track layout.

Related classes

Class Description
AliyunMixRecorder A core class that defines duet recording features, including recording, preview settings, effect settings, and callback settings.
AliyunMixMediaInfoParam A class that defines the parameters for duet recording, including the size and position of the view that is used to display the video captured by the camera and the size and position of the view that is used to display the sample video.
AliyunMixMediaRecordVideoInfo A class that defines the configurations of the video captured by the camera, including the resolution of the video.
AliyunMixMediaFileVideoInfo A class that defines the configurations of the sample video, including the file path of the sample video, the start time and end time of the sample video.

Configure duet recording

Note The camera and microphone permissions are required for duet recording. Otherwise, recording fails.

The procedure to configure duet recording is similar to that to configure video recording. The main differences lie in the input and output parameters and preview settings that you need to configure.

Configuration Procedure Description Sample code
Basic configurations 1 Create a recording instance and configure recording parameters. Configure duet recording parameters
2 Configure callbacks. Configure callbacks
3 Start preview or stop preview. Configure preview
4 Start recording or stop recording. Start recording
5 Stop recording and generate configuration information. Stop recording
Advanced configurations 6 Configure recording effects such as retouching, filters, and background music, and configure features such as photo taking. Other settings

Configure duet recording parameters

Configure duet recording parameters as the initialization parameters of the duet recording instance. For more information about the parameters that are used in the code, see Related classes.

Configure duet recording parameters

AliyunMixMediaInfoParam *mixMediaInfo = [[AliyunMixMediaInfoParam alloc] init];
mixMediaInfo.outputSizeView = previewView;
mixMediaInfo.mixVideoInfo.filePath = mixVideoFile; // The path of the sample video.

Configure layout parameters

The coordinate system of the sample video and the video captured by the camera is the same as that of the system. In this example, the PiP layout is used.
// Configure the view that is used to display the sample video. The view is displayed at the bottom in full screen.
CGFloat mixWidth = previewView.bounds.size.width;
CGFloat mixHeight = previewView.bounds.size.height;
mixMediaInfo.mixVideoInfo.frame = CGRectMake(0, 0, mixWidth, mixHeight);
mixMediaInfo.mixVideoInfo.layerLevel = 1;

// Configure the view that is used to display the video captured by the camera. The view is displayed at the 9:16 aspect ratio in portrait mode. The recording resolution is 360p. The view is displayed in the lower-right corner of the window and on the top of the sample video.
CGFloat recordRatio = 9.0 / 16.0;
mixMediaInfo.recordVideoInfo.resolution = CGSizeMake(360, 360 / recordRatio); 
CGFloat recordHeight = previewView.bounds.size.height * 0.5;
CGFloat recordWidth = recordHeight * recordRatio;
mixMediaInfo.recordVideoInfo.frame = CGRectMake(previewView.bounds.size.width - recordWidth - 4.0, previewView.bounds.size.height - recordHeight - 4, recordWidth, recordHeight);
mixMediaInfo.recordVideoInfo.layerLevel = 2;
Configure other parameters
// Configure the border of the sample video. This setting is optional.
mixMediaInfo.mixVideoInfo.borderInfo.color = UIColor.blueColor;
mixMediaInfo.mixVideoInfo.borderInfo.width = 2;
mixMediaInfo.mixVideoInfo.borderInfo.cornerRadius = 6.0;

// Configure the border of the video captured by the camera. This setting is optional.
mixMediaInfo.recordVideoInfo.borderInfo.color = UIColor.blueColor;
mixMediaInfo.recordVideoInfo.borderInfo.width = 1;
mixMediaInfo.recordVideoInfo.borderInfo.cornerRadius = 6.0;

Create a duet recording object

When you create a duet recording object, the parameters that you need to configure are similar to the parameters for basic recording. For more information, see Basic recording.
AliyunMixRecorder *_recorder = [[AliyunMixRecorder alloc] initWithMediaInfo:mixMediaInfo outputSize:CGSizeMake(720, 720)];
_recorder.outputType = AliyunIRecorderVideoOutputPixelFormatType420f;
_recorder.useFaceDetect = YES;
_recorder.faceDetectCount = 2;
_recorder.faceDectectSync = NO;
_recorder.frontCaptureSessionPreset = AVCaptureSessionPreset1280x720;
_recorder.GOP = 250;
_recorder.videoQuality = AliyunVideoQualityHight;
_recorder.recordFps = 30;
_recorder.outputPath = [taskPath stringByAppendingPathComponent:@"output.mp4"];
_recorder.cameraRotate = 0;
_recorder.beautifyStatus = YES;
_recorder.frontCameraSupportVideoZoomFactor = YES;

self.aliyunMixRecorder = _recorder;
Configure the recording duration
Note The duration of the output video is the same as the video that has the longest duration. For example, if the duration of the video captured by the camera is 10 seconds, and the duration of the sample video is 5 seconds, the duration of the duet is 10 seconds.
// We recommend that you set the duration of the sample video as the duration of the duet. If the duration is reached, a recorderDidStopWithMaxDuration callback is triggered.
AVURLAsset *asset = [AVURLAsset assetWithURL:[NSURL fileURLWithPath:[self resourcePath:mixVideoPath]]];
[self.aliyunMixRecorder setRecordMaxDuration:asset.aliyunVideoDuration];
[self.aliyunMixRecorder setRecordMinDuration:1.0];
Set a background for the duet
// Set the background color for the duet.
[self.aliyunMixRecorder setBackgroundColor:0xFF0100];

// Set the background image for the duet.
NSString *imgPath = [self.class resourcePath:@"pig.jpeg"];
[self.aliyunMixRecorder setBackgroundImageFilePath:imgPath imageDisplayMode:AliyunMixVideoBackgroundImageModeScaleAspectFit];

Configure the output audio

You can remove the recording echo, mute the video, and specify a single audio track or two mixed audio tracks for the video.
Note If the output video is muted or only the audio of the sample video is used, echo removal is not required.
// Configure the hardware echo removal effect. We recommend that you use the Hardware mode.
self.aliyunMixRecorder.recorderAECType = AliyunIRecorderAECTypeHardware;

// Configure a duet to use the recorded audio track.
[self.aliyunMixRecorder setMixAudioSource:MixAudioSourceTypeBoth];
[self.aliyunMixRecorder setMixAudioOriginalWeight:50 recordWeight:50];

Configure callbacks

You can configure callbacks to obtain the processing progress, and status of audio and video in a timely manner. For more information about the parameters that are used in the code, see Related classes.

- (void)recorderDeviceAuthorization:(AliyunIRecorderDeviceAuthor)status {
    dispatch_async(dispatch_get_main_queue(), ^{
        if (status == AliyunIRecorderDeviceAuthorAudioDenied) {
            [DeviceAuthorization openSetting:@"You are not authorized to use the microphone."];
        } else if (status == AliyunIRecorderDeviceAuthorVideoDenied) {
            [DeviceAuthorization openSetting:@"You are not authorized to use the camera."];
        }
    });
}

// The recording progress.
- (void)recorderVideoDuration:(CGFloat)duration {
    NSLog(@"Mix Record Video Duration: %f", duration);
}

// Stop recording.
- (void)recorderDidStopRecording {
    NSLog(@"Mix Record Stop Recording");
}

// The recording stops when the maximum recording duration is reached.
- (void)recorderDidStopWithMaxDuration {
    NSLog(@"Mix Record Stop Recording With Max Duration");
    [self.aliyunMixRecorder finishRecording]; // The recording is complete.
}

// The recording is complete.
- (void)recorderDidFinishRecording {
    NSLog(@"Mix Record Did Finish Recording");
}

// The merge for duet recording is complete.
- (void)mixRecorderComposerDidComplete {
    NSLog(@"Mix Record Complete");
    [[MBProgressHUD HUDForView:self.view] hideAnimated:YES];
}

// An error occurred during the merge.
- (void)mixRecorderComposerDidError:(int)errorCode {
    NSLog(@"Mix Record Error");
    [[MBProgressHUD HUDForView:self.view] hideAnimated:YES];
}

// Merge starts for duet recording.
- (void)mixRecorderComposerDidStart {
    NSLog(@"Mix Record Start");
    MBProgressHUD *hud = [MBProgressHUD showHUDAddedTo:self.view animated:YES];
    hud.mode = MBProgressHUDModeDeterminate;
    hud.removeFromSuperViewOnHide = YES;
    hud.label.text = @"Merging...";
}

// The merge is complete.
- (void)mixRecorderComposerOnProgress:(CGFloat)progress {
    NSLog(@"Mix Record Progress: %f", progress);
    MBProgressHUD *hub = [MBProgressHUD HUDForView:self.view];
    hub.progress = progress / self.aliyunMixRecorder.recordDuration;
}

Configure preview

For more information about the parameters that are used in the code, see Related classes.

// Enable the preview.
[self.aliyunMixRecorder startPreviewWithPositon:AliyunIRecorderCameraPositionFront];

// Stop the preview. Call stopPreview after the recording is complete.
[self.aliyunMixRecorder stopPreview];

Start recording

startRecording and stopRecording must be called in pairs. You can call them once or multiple times, and one or more temporary video clips are generated. For more information about the parameters that are used in the code, see Related classes.

// Start recording a video clip.
[self.aliyunMixRecorder startRecording];  

// Stop recording a video clip.
[self.aliyunMixRecorder stopRecording];

Stop recording

After the recording is complete, you can call finishRecording to merge the sample video and the video captured by the camera. For more information about the parameters that are used in the code, see Related classes.

// Stop recording, and merge the sample video and the video captured by the camera.
[self.aliyunMixRecorder finishRecording];

Other settings

Duet recording also supports recording effects such as retouching, filters, and background music. You can use the AliyunMixRecorder class to configure recording effects and features such as photo taking. The configuration method is similar to the configuration method that is used for basic recording. For more information, see Basic recording.