Auto Scaling can automatically add or remove Elastic Compute Service (ECS) instances in a scaling group, but it cannot directly update the IP address whitelist of a Tair (Redis OSS-compatible) instance. This means that when a scale-out adds a new ECS instance, the instance cannot connect to Redis until you manually add its private IP address to the whitelist — and the reverse applies for scale-in.
This tutorial shows how to eliminate that manual step by combining a lifecycle hook with the public CloudOps Orchestration Service (OOS) template ACS-ESS-LifeCycleModifyRedisIPWhitelist. When a scaling event occurs, the lifecycle hook pauses the instance in a Pending Add state and triggers OOS to update the IP address whitelist automatically, then completes the lifecycle action.
How it works
Scale-out event triggered
↓
Auto Scaling adds ECS instance → instance enters Pending Add state
↓
Lifecycle hook fires → OOS executes ACS-ESS-LifeCycleModifyRedisIPWhitelist
↓
OOS calls ecs:DescribeInstances to get the private IP address
↓
OOS calls kvstore:ModifySecurityIps to add the IP to the whitelist
↓
OOS calls ess:CompleteLifecycleAction → instance moves to InServiceFor scale-in, create a separate lifecycle hook for scale-in purposes and apply the same ACS-ESS-LifeCycleModifyRedisIPWhitelist template.
Prerequisites
Before you begin, ensure that you have:
A scaling group in the Enabled state
A Tair (Redis OSS-compatible) instance
A Resource Access Management (RAM) role for OOS with Alibaba Cloud Service as the trusted entity and CloudOps Orchestration Service as the trusted service — the example in this tutorial uses OOSServiceRole
Step 1: Grant OOS the required permissions
The ACS-ESS-LifeCycleModifyRedisIPWhitelist template needs permissions to describe ECS instances, modify the Redis IP address whitelist, and complete lifecycle actions. Create a permission policy with these permissions and attach it to the OOSServiceRole RAM role.
Create the permission policy
Log on to the RAM console.
In the left-side navigation pane, choose Permissions > Policies.
Click Create Policy.
On the Create Policy page, click the JSON tab and enter the following policy document:
{ "Version": "1", "Statement": [ { "Action": [ "ecs:DescribeInstances" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "kvstore:ModifySecurityIps" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "ess:CompleteLifecycleAction" ], "Resource": "*", "Effect": "Allow" } ] }Set Name to
ESSHookPolicyForRedisWhitelistand click OK.
Attach the policy to OOSServiceRole
In the left-side navigation pane, choose Identities > Roles.
Find the OOSServiceRole RAM role and click Grant Permission in the Actions column.
In the Grant Permission panel, set Resource Scope to Account and select the ESSHookPolicyForRedisWhitelist custom policy.
Click Grant permissions.
Step 2: Create a lifecycle hook and trigger a scale-out
Create the lifecycle hook
Log on to the Auto Scaling console.
In the left-side navigation pane, click Scaling Groups.
In the top navigation bar, select the region where your scaling group is located.
Find the scaling group and click its ID in the Scaling Group Name/ID column, or click Details in the Actions column.
Click the Lifecycle Hook tab, then click Create Lifecycle Hook.
Configure the following parameters and click OK:
Parameter Value Name ESSHookForAddRedisWhitelistScaling Activity Scale-out Event Timeout Period 300 (seconds). Configure this parameter based on your business requirements. We recommend that you estimate the period of time required to perform custom operations on instances and set this value accordingly. If the timeout period is shorter than the actual execution time, the O&M tasks may fail. Default Execution Policy Continue Send Notification When Lifecycle Hook Takes Effect Select OOS Template > Public Templates > ACS-ESS-LifeCycleModifyRedisIPWhitelistIn the
ACS-ESS-LifeCycleModifyRedisIPWhitelisttemplate, configure the following parameters:Parameter Value dbInstanceId The ID of the Tair instance modifyMode Append— adds the private IP address to the whitelist during scale-outOOSAssumeRole OOSServiceRole
Lifecycle hooks do not take effect when you manually add or remove ECS instances to or from a scaling group. They only fire during automated scaling events triggered by scaling rules, scheduled tasks, or event-triggered tasks.
Trigger a scale-out event
Create and execute a scaling rule to trigger the lifecycle hook.
Click the Scaling Rules and Event-triggered Tasks tab.
On the Scaling Rules tab, click Create Scaling Rule.
Configure the following parameters and click OK:
Parameter Value Rule Name Add1Rule Type Simple Scaling Rule Operation Add 1 Instances Find the Add1 scaling rule and click Execute in the Actions column.
In the Execute Scaling Rule dialog box, click OK.
After the rule executes, Auto Scaling adds one ECS instance to the scaling group. The instance enters the Pending Add state while the lifecycle hook is active. During the timeout period, Auto Scaling notifies OOS to run the O&M tasks defined in the ACS-ESS-LifeCycleModifyRedisIPWhitelist template.
Step 3: Verify the IP address whitelist
Log on to the Tair (Redis OSS-compatible) console.
In the left-side navigation pane, click Instances.
Find the instance and click its ID in the Instance ID/Name column.
In the left-side navigation pane, click Whitelist Settings.
If the private IP address of the newly added ECS instance appears in the whitelist, the lifecycle hook and OOS template are working as expected.
If the IP address is missing, go to the OOS console to check the execution logs (see Step 4: View the OOS execution).
Step 4: View the OOS execution (optional)
Log on to the OOS console.
In the left-side navigation pane, choose Automated Task > Task Execution Management.
Find the execution by time and click Details in the Actions column.
In the Basic Information section, check the execution ID and status. In the Execution Steps and Results section, click a task node to view details.
If the execution fails, an error message is displayed on the execution details page. For more information, see View the details of an execution.
FAQ
If an Operation and Maintenance (O&M) task fails, find the cause based on the error message in the execution result. For more information, see FAQ.
Common error messages are as follows:
Error message | Cause | Solution |
Forbidden.Unauthorized message: A required authorization for the specified action is not supplied. | Auto Scaling is not authorized to perform the specified action. | Verify that the required permissions are granted to the OOSServiceRole RAM role. |
Forbidden.RAM message: User not authorized to operate on the specified resource, or this API doesn't support RAM. | The RAM user or RAM role does not have the permissions to operate on the corresponding resource. | Verify that the OOSServiceRole RAM role has the required permissions. For example, you can grant the sample permissions for the OOS service to the RAM role. You must add operation permissions to the RAM role to ensure that the OOS service can manage the resources specified in the OOS template. |
LifecycleHookIdAndLifecycleActionToken.Invalid message: The specified lifecycleActionToken and lifecycleActionId you provided does not match any in process lifecycle action. | The ongoing lifecycle action has expired or has been aborted. | Evaluate the timeout period of the lifecycle hook to ensure that the O&M tasks defined in the OOS template can be completed within the timeout period. |
Troubleshooting
| Error message | Cause | Solution |
|---|---|---|
Forbidden.Unauthorized message: A required authorization for the specified action is not supplied. | OOS does not have permission to perform the action. The OOSServiceRole RAM role is missing one or more required permissions. | Verify that the ESSHookPolicyForRedisWhitelist policy is attached to OOSServiceRole and contains all three required actions (ecs:DescribeInstances, kvstore:ModifySecurityIps, ess:CompleteLifecycleAction). |
Forbidden.RAM message: User not authorized to operate on the specified resource, or this API doesn't support RAM. | The RAM role does not have the permissions to operate on the corresponding resources. | Check the ESSHookPolicyForRedisWhitelist policy document. Make sure the Resource field is set to * for all three actions, and that the policy is attached to OOSServiceRole. |
LifecycleHookIdAndLifecycleActionToken.Invalid message: The specified lifecycleActionToken and lifecycleActionId you provided does not match any in process lifecycle action. | The lifecycle hook action already ended or was stopped before OOS completed the O&M tasks. | Assess the Timeout Period of the lifecycle hook to make sure that the O&M tasks specified in the OOS template can be completed within the allotted time limit. |
What's next
To automate IP address removal during scale-in, create a separate lifecycle hook with Scaling Activity set to Scale-in Event and apply the same
ACS-ESS-LifeCycleModifyRedisIPWhitelisttemplate.To learn more about lifecycle hooks, see Lifecycle hooks.
To learn more about OOS templates, see OOS template overview.