Fix Screen Rotation Sync for Android & iOS SDKs - Cloud Phone

In cloud phone scenarios, users operate a virtual device that runs in the cloud. The local client acts only as a medium for display and interaction. When the cloud phone's screen orientation changes, such as from portrait to landscape, the user experience is severely affected if the local client does not respond in sync. This can cause display issues such as inverted images, stretching, or black bars, and interaction misalignments. This topic describes how to implement a precise synchronization mechanism using Android and iOS client SDKs. This mechanism synchronizes the local screen orientation with rotation instructions from the cloud. It ensures that the local interface and video stream always match the cloud device's screen orientation. This provides an immersive and seamless remote operation experience.

Background

Traditional mobile applications usually rely on device sensors or system configurations to automatically respond to screen rotation. However, in a cloud phone architecture, the screen orientation must be driven by the state of the cloud's virtual device, not the orientation of the local physical device. If you enable automatic system rotation directly, the following problems occur:

The local rotation is not synchronized with the cloud's state, which causes incorrect screen display.
Users accidentally rotating their phones can trigger rotation, which interferes with normal operations.
The video rendering layer, such as SurfaceView, TextureView, or a Metal/OpenGL View, does not adapt to the new orientation. This results in cropping or distortion.

Therefore, you must take control of rotation. The cloud sends specific orientation instructions. The local client then forces an interface orientation switch and adjusts the video rendering logic accordingly.

Solution overview

This solution implements the following core capabilities on the Android and iOS platforms:

Unified mechanism to receive rotation instructions
- Receives rotation commands from the cloud through a custom data channel, such as wy_vdagent_default_dc.
- Uses a standardized protocol to parse rotation parameters. For example, rotation = 0/1/3 corresponds to portrait, landscape left, and landscape right, respectively.
Forced synchronization of local interface orientation
- Android: Calls setRequestedOrientation() to dynamically lock the Activity orientation.
- iOS: Overrides supportedInterfaceOrientations and uses setNeedsUpdateOfSupportedInterfaceOrientations (iOS 16+) or a private API to switch the orientation.
- Both platforms disable automatic device rotation (shouldAutorotate = false or disable sensor response) to ensure they only respond to cloud instructions.
Precise adaptation of the video rendering layer
- Android: Uses TextureView and calls setSurfaceRotation() to directly rotate the underlying texture. This avoids image distortion.
- iOS: Rotates the StreamView using CGAffineTransform and dynamically adjusts its centroid. This ensures the image is centered and has the correct aspect ratio.
Support for multiple orientations
- Supports three orientations: Portrait, Landscape Left, and Landscape Right.
- Correctly handles differences in the Home button position to improve interaction consistency on iOS devices in landscape mode.

Implementation steps

Handle local rotation using the Android SDK

// Disable automatic rotation for the cloud phone.
bundle.putBoolean(StreamView.CONFIG_DISABLE_ORIENTATION_CLOUD_CONTROL, true);
// Use TextureView.
mStreamView = findViewById(R.id.stream_view);
mStreamView.enableTextureView(true);
// Handle rotation.
mStreamView.getASPEngineDelegate().addDataChannel(new DataChannel("wy_vdagent_default_dc") {
            @Override
            protected void onReceiveData(byte[] buf) {
                String str = "";
                try {
                    str = new String(buf, "UTF-8");
                } catch (UnsupportedEncodingException e) {
                    str = new String(buf);
                }
                Log.i(TAG, "wy_vdagent_default_dc dc received " + buf.length + " bytes data:" + str);
                CommandUtils.parseCommand(str, new CommandUtils.CommandListener() {
                    @Override
                    public void onCameraAuthorize() {
                        checkStartCpd();
                    }
                    @Override
                    public void onRotation(int rotation) {
                        runOnUiThread(() -> {
                            mStreamView.setSurfaceRotation(rotation);
                            if (rotation == 1) {
                                setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
                            } else if (rotation == 3) {
                                setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_REVERSE_LANDSCAPE);
                            } else {
                                setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
                            }
                        });
                    }
                    @Override
                    public void onUnknownCommand(String cmd) {
                        showError("Unknown command: " + cmd);
                    }
                });
            }
            @Override
            protected void onConnectStateChanged(DataChannelConnectState state) {
                Log.i(TAG, "wy_vdagent_default_dc dc connection state changed to " + state);
            }
        });

Handle local rotation using the iOS SDK

In list.info, configure Supported interface orientations to enable application support for portrait, landscape left, and landscape right orientations.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
	<string>UIInterfaceOrientationPortrait</string>
	<string>UIInterfaceOrientationLandscapeLeft</string>
	<string>UIInterfaceOrientationLandscapeRight</string>
</array>
</plist>

Create BaseViewController to implement the rotation logic.

@implementation BaseViewController {
    NSInteger mRoration;
}

- (void)viewDidLoad {
    [super viewDidLoad];
    self.view.backgroundColor = [UIColor whiteColor];
}

- (void)switchRoration:(NSInteger)roration {
    mRoration = roration;
    // Force rotation to landscape (recommended for iOS 16+).
    if (@available(iOS 16.0, *)) {
        [self setNeedsUpdateOfSupportedInterfaceOrientations];
    } else {
        // Legacy method (deprecated, but compatible).
        NSNumber *value = @([self supportedInterfaceOrientations]);
        [[UIDevice currentDevice] setValue:value forKey:@"orientation"];
    }
}

- (UIInterfaceOrientationMask)supportedInterfaceOrientations {
    switch (mRoration) {
        case 1:
            return UIInterfaceOrientationMaskLandscapeLeft;
        case 3:
            return UIInterfaceOrientationMaskLandscapeRight;
        default:
            return UIInterfaceOrientationMaskPortrait;
    }
}

- (BOOL)shouldAutorotate {
    return false; // Allow automatic rotation to supported orientations.
}

// Optional: Specify the preferred orientation for presentation.
- (UIInterfaceOrientation)preferredInterfaceOrientationForPresentation {
    switch (mRoration) {
        case 1:  return UIInterfaceOrientationLandscapeLeft;
        case 3:  return UIInterfaceOrientationLandscapeRight;
        default: return UIInterfaceOrientationPortrait;
    }
}

@end

Establish a path for cloud instructions to reach the local UI and rendering layer.

self.esdAgent = [[DemoEDSAgentChannel alloc] initWithParameter:DATA_CHANNEL_NAME];
self.esdAgent.streamView = self.streamView;
self.esdAgent.viewController = self;
[self.streamView addDataChannel:self.esdAgent];

Listen to and respond to cloud phone rotation instructions.

@interface DemoEDSAgentChannel() <CommandListener>

@property (nonatomic, assign) CGRect rect;

@end

@implementation DemoEDSAgentChannel

- (void)setViewController:(BaseViewController *)viewController {
    self.rect = viewController.view.bounds;
    _viewController = viewController;
}

- (void)onConnectStateChanged:(ASPDCConnectState)state {
    NSLog(@"[DemoEDSAgentChannel] onConnectStateChanged %ld", state);
    if (state == OPEN) {
        // to send data
    }
}
    
- (void)onReceiveData:(NSData * _Nonnull)buf {
    NSString *string = [[NSString alloc] initWithData:buf encoding:NSUTF8StringEncoding];
    NSLog(@"[DemoEDSAgentChannel] onConnectStateChanged %@", string);
    [CommandUtils parseCommand:string listener:self];
}

#pragma mark - CommandListener
- (void)onRotation:(NSInteger)value {
    NSLog(@"[DemoEDSAgentChannel] onRotation %ld", value);
    dispatch_async(dispatch_get_main_queue(), ^{
        [self.viewController switchRoration:value];
        if (value == 1 || value == 3) {
            self.streamView.center = CGPointMake(CGRectGetMidY(self.viewController.view.bounds),
                                                 CGRectGetMidX(self.viewController.view.bounds));
            self.streamView.transform = CGAffineTransformMakeRotation(-M_PI_2*value) ;
        } else {
            self.streamView.center = CGPointMake(CGRectGetMidX(self.rect),
                                                 CGRectGetMidY(self.rect));
            self.streamView.transform = CGAffineTransformIdentity;
        }
    });
}