A cluster, especially a subscription cluster, can have new third-party software installed to modify the operating environment of the cluster. After a cluster is created, the cluster script allows you to select nodes in batches and run your specified script to fulfill individual requirements.
A cluster script is similar to a bootstrap action. After creating a cluster, you can install software packages to your cluster, for example:
Use Yum to install the software that has been provided.
Directly download public software packages from the public network.
Read your data from OSS.
Install and run a service like Flink or Impala, but the script to be compiled is more complex.
We strongly recommend that you test the cluster script on a single node first. After script verification, you can perform operations on the whole cluster.
A cluster script can run on an idle or running cluster. On the cluster list page, click Display details of the corresponding cluster.
Click Cluster script on the left-side menu to enter the cluster script execution interface. A list of cluster scripts that have been run is on the right side.
Click Create and run at the upper right corner to enter the creation interface.
Input configuration items in the creation interface. Select the node for execution and click Run to confirm and run the operations.
You can see the newly created cluster script in the cluster script list. Click Refresh to update the cluster script status.
You can click Display details to display the running status of a script on each node or click Refresh to update the running status of a cluster on each node.
The cluster script can only run on available clusters that are idle or running. The cluster script is applicable for long-standing clusters. For on-demand clusters, perform a bootstrap action to initialize the clusters.
The cluster script downloads a script from the OSS and run it on specified node. If the returned value is 0, the execution has failed. If the execution fails, you can log on to the node to check the running log. The running log for each node is located at /var/log/cluster-scripts/clusterScriptId. If the cluster is configured with an OSS log directory, the running log is also uploaded to osslogpath/clusterId/ip/cluster-scripts/clusterScriptId.
By default, the root account is used to run the specified script. In the script, you can use su hadoop to switch to Hadoop account.
A cluster script can successfully run on some nodes, but fails on others. For example, the restart of a node can lead to a failure in script operation. After resolving the error, you can run the cluster script again. After a cluster is expanded, you can specify the expanded node for separate execution of the cluster script.
Only one cluster script at a time can run on a cluster. If a cluster script is running, you cannot submit a new cluster script for execution. For each cluster, you can retain at most ten cluster script records. Therefore, if you already have ten records, to create a new cluster script, you must delete the previous records first.
For a script similar to a bootstrap action, you can specify the file in the script to be downloaded from OSS. In the following example, the file oss://yourbucket/myfile.tar.gz is downloaded and decompressed to the directory /yourdir:
osscmd --id=<yourid> --key=<yourkey> --host=oss-cn-hangzhou-internal.aliyuncs.com get oss://<yourbucket>/<myfile>.tar.gz ./<myfile>.tar.gz
mkdir -p /<yourdir>
tar -zxvf <myfile>.tar.gz -C /<yourdir>
OSSCMD is pre-installed on the node and can be called directly to download the file.
The OSS host address can be an intranet address, an Internet address, or a VPC network address. If a classic network is used, you must specify an intranet address. If the network is located in Hangzhou, the intranet address is oss-cn-hangzhou-internal.aliyuncs.com. If a VPC network is used, you must specify a domain name that can be accessed from the VPC intranet. If the network is located in Hangzhou, the domain name is vpc100-oss-cn-hangzhou.aliyuncs.com.
Additional system software packages can be installed to the script using Yum, for example, ld-linux.so.2:
yum install -y ld-linux.so.2