Apsara Video SDK:Multi-source recording

Supported editions

Terms

This topic involves some special terms. To facilitate your understanding, we recommend that you familiarize yourself with terms such as multi-source recording, track, and track layout. For more information, see Terms.

Related classes

Multi-source recording process

Initialization

Initialize the AliyunRecorder class, create a recording instance, and configure recording parameters.

Configure output parameters

Use the AliyunRecorderVideoConfig class to configure parameters of the output video, including image parameters and compression parameters. Image parameters include the resolution, frame rate, rotation angle, and padding mode. Compression parameters include the encoding format, GOP size, bitrate, and video quality. In most cases, you do not need to configure the compression parameters, unless otherwise required.

AliyunRecorderVideoConfig *videoConfig = [[AliyunRecorderVideoConfig alloc] init];
videoConfig.resolution = CGSizeMake(720, 1280); // 720P
videoConfig.fps = 30;
...

Add input sources

Create a configuration object by specifying the outputPath parameter in the parameter output object, and then add input sources.

// Create a configuration object.
AliyunRecorderConfig *config = [[AliyunRecorderConfig alloc] initWithVideoConfig:videoConfig outputPath:[taskPath stringByAppendingPathComponent:@"output.mp4"]];


// Add a camera recording source.
// Specify the camera layout parameters.
AliyunVideoRecordLayoutParam *cameraLayout = [[AliyunVideoRecordLayoutParam alloc] initWithRenderMode:AliyunRenderMode_ResizeAspectFill];
cameraLayout.size = resolution;
cameraLayout.center = CGPointMake(resolution.width / 2.0, resolution.height / 2.0);
cameraLayout.zPosition = 1;
// Add a recording source and return the controller.
self.cameraRecorderController = [config addCamera:cameraLayout];
self.cameraRecorderController.preview = self.videoView;
self.cameraRecorderController.camera.resolution = AliyunRecordCameraResolution_3840x2160;
self.cameraRecorderController.camera.position = AVCaptureDevicePositionBack;


// Add a screen recording source.
// Specify the layout parameters for screen recording.
AliyunVideoRecordLayoutParam *viewRecordLayout = [[AliyunVideoRecordLayoutParam alloc] initWithRenderMode:AliyunRenderMode_ResizeAspect];
viewRecordLayout.size = CGSizeMake(0.5*videoConfig.resolution.width, 0.5*videoConfig.resolution.height);
viewRecordLayout.center = CGPointMake(0.5*videoConfig.resolution.width, 0.5*videoConfig.resolution.height);
viewRecordLayout.zPosition = 2;
// Specify the parameters for the screen recording source.
AliyunViewRecordSource *viewSource = [[AliyunViewRecordSource alloc] initWithTargetView:drawView fps:videoConfig.fps];
viewSource.captureInBackground = YES;
// Add a recording source and return the controller.
self.viewRecordController = [config addViewSource:viewSource layout:viewRecordLayout];


// Add a local playback source.
// Specify the layout parameters for screen recording.
AliyunVideoRecordLayoutParam *playerRecordLayout = [[AliyunVideoRecordLayoutParam alloc] initWithRenderMode:AliyunRenderMode_ResizeAspect];
playerRecordLayout.size = CGSizeMake(0.5*videoConfig.resolution.width, 0.5*videoConfig.resolution.height);
playerRecordLayout.center = CGPointMake(0.5*videoConfig.resolution.width, 0.5*videoConfig.resolution.height);
playerRecordLayout.zPosition = 2;
// Specify the parameters for the screen recording source.
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:videoPath] options:nil];
AliyunPlayerRecordSource *playerSource = [[AliyunPlayerRecordSource alloc] initWithAsset:asset fps:videoConfig.fps];
// Add a recording source and return the controller.
self.playerRecordController = [config addMVSource:playerSource layout:playerRecordLayout];
self.playerRecordController.preview = self.playerView;

Create an instance

// Specify the parameters to create an instance.
AliyunRecorder *recorder = [[AliyunRecorder alloc] initWithConfig:config];
recorder.delegate = self;
recorder.clipManager.maxDuration = 5;
recorder.clipManager.minDuration = 1;
recorder.clipManager.deleteVideoClipsOnExit = YES;
self.aliyunRecorder = recorder;

Configure preview

// Enable preview.
[self.aliyunRecorder startPreview];

// After the recording is complete, call the stopPreview method to stop preview.
[self.aliyunRecorder stopPreview];

Start recording

The startRecord and stopRecord methods must be called in pairs. You can call the methods once or multiple times, and one or more temporary video clips are generated.

// Start recording a video clip.
[self.aliyunRecorder startRecord]; 

// Stop recording a video clip. After you call this method, recording does not immediately stop. Instead, the recording state changes. You can invoke the onAliyunRecorder:stateDidChange: callback to obtain the recording state.
[self.aliyunRecorder stopRecord];

The following table describes the recording states.

Stop recording

If you stop recording, the recorded video clips are merged into a video or the configuration information of the recorded video clips is generated.

// Stop recording and merge the recorded video clips into a video.
[self.aliyunRecorder finishRecord:^(NSString *outputPath, NSError *error) {
 if (!error) {
 // After recording is complete, you can preview, edit, and upload the recorded video stored in the output path.
 }
}];

// Stop recording without merging the recorded video clips into a video. In this case, the taskPath parameter is returned.
[self.aliyunRecorder finishRecordForEdit:^(NSString *taskPath, NSError *error) {
 if (!error) {
 // After recording is complete, you can configure the taskPath parameter to initialize AliyunEditor and edit the video. For more information, see topics about video editing.
 }
}];

Input source control: Camera recording control

After you add a source, the controller that is based on the AliyunCameraRecordController protocol is returned. In this case, you can use the controller to configure parameters, adjust the border, use basic retouching, apply face stickers, static stickers, animated GIFs, static filters, and animated filters, and take photos.

Configure common parameters

// Specify the flashlight mode.
self.cameraRecorderController.camera.torchMode = AVCaptureTorchModeOn;

// Specify the camera position.
self.cameraRecorderController.camera.position = AVCaptureDevicePositionBack;

// Specify the zoom factor.
self.cameraRecorderController.camera.videoZoomFactor = 2.0;

// Specify the exposure level.
self.cameraRecorderController.camera.exposureValue = 0.8;

// Specify the orientation.
self.cameraRecorderController.camera.orientation = UIDeviceOrientationLandscapeLeft;

// Specify the resolution. We recommend that you do not change the value of this parameter. The resolution is automatically adapted based on the output size.
self.cameraRecorderController.camera.resolution = AliyunRecordCameraResolution_1280x720;

// Specify the flash mode. If you set the flash mode to On, the flash is turned on when you take photos.
self.cameraRecorderController.camera.flashMode = AVCaptureFlashModeOn;

Border adjustment

AliyunVideoRecordBorderInfo *cameraBorder = [AliyunVideoRecordBorderInfo new];
cameraBorder.color = UIColor.whiteColor;
cameraBorder.width = 3.0;
cameraBorder.cornerRadius = 10.0;
self.cameraRecorderController.borderInfo = cameraBorder;

Basic retouching

// Enable basic retouching.
self.cameraRecorderController.beautifyStatus = YES;
self.cameraRecorderController.beautifyValue = 80;

// Disable basic retouching.
self.cameraRecorderController.beautifyStatus = NO;
self.cameraRecorderController.beautifyValue = 0;

Face stickers

// Add a face sticker.
[self.cameraRecorderController applyFaceSticker:[self.class resourcePath:@"Gif/hanfumei-800"]];

Filters

You can create custom filters. For more information, see Filters and transitions.

// Add a filter.
AliyunEffectFilter *filter = [[AliyunEffectFilter alloc] initWithFile:path];
[self.cameraRecorderController applyFilter:filter];

Animated filters

// Add an animated filter.
NSString *filterDir = [self.class resourcePath:@"AnimationEffect/split_screen_3"];
AliyunEffectFilter *animationFilter =[[AliyunEffectFilter alloc] initWithFile:filterDir];
[self.cameraRecorderController applyAnimationFilter:animationFilter];

Static stickers

// Add a static sticker.
AliyunImageStickerController *imageController = [self.cameraRecorderController addImageSticker:imagePath];
[imageController beginEdit];
imageController.image.center = CGPointMake(150, 200);
[imageController endEdit];

Animated stickers

You can create custom animated stickers. For more information, see Animated stickers.

// Add an animated sticker.
AliyunGifStickerController * gifController = [self.cameraController addGifStickerWithConfig:[PathTool boundlePathWithPath:@"Resource/Gif/hanfumei-800"]];
[gifController beginEdit];
gifController.gif.center = CGPointMake(150, 200);
[gifController endEdit];

Photo capture

// Take a photo and asynchronously obtain the following types of images:
 // image: the rendered image that is collected.
 // rawImage: the raw image that is collected.
[self.cameraRecorderController takePhoto:^(UIImage *image, UIImage *rawImage) {
 
}];

Input source control: Screen recording control

After you add a source, the controller that is based on the AliyunViewRecordController protocol is returned. In this case, you can use the controller to only adjust the border.

Borders

AliyunVideoRecordBorderInfo *border = [AliyunVideoRecordBorderInfo new];
border.color = UIColor.whiteColor;
border.width = 3.0;
border.cornerRadius = 10.0;
self.viewRecorderController.borderInfo = border;

Input source control: Local video control

After you add a source, the controller that is based on the AliyunPlayerRecordController protocol is returned. In this case, you can use the controller to only adjust the border.

Borders

AliyunVideoRecordBorderInfo *border = [AliyunVideoRecordBorderInfo new];
border.color = UIColor.whiteColor;
border.width = 3.0;
border.cornerRadius = 10.0;
self.playerRecordController.borderInfo = border;

Other features

Features such as background music, watermarks, and background settings are supported.

Manage video clips

Call the startRecord and stopRecord methods to record multiple video clips. You can use the clipManager class to manage the video clips. For example, you can delete the last video clip or delete all video clips. For more information about parameters of the class, see AliyunClipManager.

// Delete the last video clip.
[self.aliyunRecorder.clipManager deletePart];
// Delete all video clips.
[self.aliyunRecorder.clipManager deleteALLPart];

// Obtain the total number of video clips.
[self.aliyunRecorder.clipManager partCount];

Speed ramping

// Specify the recording speed. We recommend that you set this parameter to a value from 0.5 to 2. 
[self.aliyunRecorder setRate:2];

Background music

// Add background music.
AVURLAsset *audioAsset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:filePath] options:nil];
float audioDuration = CMTimeGetSeconds(audioAsset.duration);
[self.aliyunRecorder.config setBgMusicWithFile:filePath
 startTime:0.0 // The background music starts from the 0-second position.
 duration:MIN(self.aliyunRecorder.clipManager.maxDuration, audioDuration)];

// Remove the background music.
[self.aliyunRecorder.config removeBgMusic];

Watermarks

// Generate a watermark.
- (AliyunRecorderImageSticker *) waterMark
{
 if (!_waterMark) {
 NSString *watermarkPath = [AlivcImage pathOfImageName:@"shortVideo_paster_gif"];
 _waterMark = [[AliyunRecorderImageSticker alloc] initWithImagePath:watermarkPath];
 _waterMark.size = CGSizeMake(42, 30);
 _waterMark.center = CGPointMake(_waterMark.size.width * 0.5 + 4, _waterMark.size.height * 0.5 + 4);
 _waterMark.autoresizingMask = UIViewAutoresizingFlexibleRightMargin | UIViewAutoresizingFlexibleBottomMargin;
 }
 return _waterMark;
}

// Apply the watermark.
[self.aliyunRecorder.config addWaterMark:self.waterMark];

// Remove the watermark.
[self.aliyunRecorder.config removeWaterMark:self.waterMark.stickerId];

Background settings

// Generate a background object.
- (AliyunRecorderBackgroundInfo *) bgInfo
{
 if (!_bgInfo) {
 _bgInfo = [AliyunRecorderBackgroundInfo new];
 // _bgInfo.color = UIColor.redColor;
 NSString *imgName = @"xxx.png";
 _bgInfo.image = [UIImage imageNamed:imgName];
 _bgInfo.renderMode = AliyunRenderMode_ResizeAspectFill;
 }
 return _bgInfo;
}

// Apply the background object.
self.aliyunRecorder.config.bgInfo = self.bgInfo;

Custom rendering

After you add a camera recording source, the short video SDK passes the CMSampleBufferRef data that is collected by the camera to the business layer by using the customRender callback. You can import a third-party SDK to the business layer to perform custom rendering, and then return the rendering result CVPixelBufferRef to the short video SDK for preview and production.

// Step 1: Configure custom rendering. Take note that a camera recording source must be added.
self.aliyunRecorder.customRender = self;
// Step 2: Implement the AliyunRecorderCustomRender protocol.
- (CVPixelBufferRef) onAliyunRecorderCustomRenderToPixelBuffer:(AliyunRecorder *)recorder withSampleBuffer:(CMSampleBufferRef)sampleBuffer {
 // Perform custom rendering. You can use Queen SDK to perform custom rendering. After the custom rendering is complete, the CVPixelBufferRef sample is returned.
 // If you do not perform custom rendering, CMSampleBufferGetImageBuffer(sampleBuffer) is returned.
 ...
}

Recording callbacks

You can configure callbacks to obtain the processing progress and status of audio and videos at the earliest opportunity.

// Configure the following common callbacks:
#pragma mark - AliyunRecorderDelegate

- (void)onAliyunRecorderWillStopWithMaxDuration:(AliyunRecorder *)recorder {
 NSLog(@"Record2 Will Stop Recording With Max Duration");
 [self.aliyunRecorder stopPreview];
}

- (void)onAliyunRecorderDidStopWithMaxDuration:(AliyunRecorder *)recorder {
 NSLog(@"Record2 Did Recording With Max Duration");
 [self.aliyunRecorder finishRecord:^(NSString *outputPath, NSError *error) {
 if (!error) {

 }
 }];
}

- (void)onAliyunRecorder:(AliyunRecorder *)recorder progressWithDuration:(CGFloat)duration {
 NSLog(@"Record2 Video Duration: %f", duration);
}

- (void)onAliyunRecorder:(AliyunRecorder *)recorder occursError:(NSError *)error {
 NSLog(@"Record2 Occurs Error: %@", error);
}