Cloud Monitor:Manage metric import tasks for cloud services of third-party providers

Install the Cloud Monitor agent for the host where the AWS or Tencent Cloud service resides.

Install an exporter for the host where the Tencent Cloud service resides.

1. Log on to the host where the Tencent Cloud service resides as the root user.

2. Run the following command to download the installation package of the exporter:

   wget https://cms-prometheus.oss-cn-hangzhou.aliyuncs.com/tencentyun/qcloud-exporter-linux-amd64.tar.gz

3. Run the following command to decompress the qcloud-exporter-linux-amd64.tar.gz file:

   tar zxvf qcloud-exporter-linux-amd64.tar.gz

4. Run the following commands to view all files in the qcloud-exporter-linux-amd64 directory:

   cd qcloud-exporter-linux-amd64

   ll qcloud-exporter-linux-amd64

   qcloud-cdb-product.yml   qcloud-clb-product.yml     qcloud-cvm-product.yml  qcloud_exporter                     qcloud-redis-cluster-product.yml
   qcloud-cdn-product.yml   qcloud-cmongo-product.yml  qcloud-dc-product.yml   qcloud-exporter-linux-amd64.tar.gz  qcloud-redis-product.yml
   qcloud-clb7-product.yml  qcloud-cos-product.yml     qcloud-dcx-product.yml  qcloud-nat-product.yml

   The files whose names are suffixed by .yml in the directory are the configuration files of the Tencent Cloud service. You can edit the configuration file in the directory or copy a configuration file to the directory to modify the settings.

5. Edit the qcloud-redis-product.yml configuration file.

   Note

   In this example, the qcloud-redis-product.yml configuration file is used.

   1. Run the following command to edit the qcloud-redis-product.yml configuration file:

      vi qcloud-redis-product.yml

   2. Edit the qcloud-redis-product.yml configuration file.

             credential:
               access_key: "access_key" # The AccessKey ID of the current Tencent Cloud account. 
               secret_key: "secret_key" # The AccessKey secret of the current Tencent Cloud account. 
               region: "region" # The region where the ApsaraDB for Redis instance resides. 
           
             rate_limit: 15   # The maximum number of API operations that you can call. The upper limit is 20 calls per second or 1,200 calls per minute. 
             products:
               - namespace: QCE/REDIS    # The namespace of the ApsaraDB for Redis instance. 
                all_metrics: true # The value true indicates that all metrics of ApsaraDB for Redis are monitored. 
                all_instances: true # The value true indicates that all instances of ApsaraDB for Redis are monitored. 
                #only_include_metrics: []
                #only_include_instances: [crs-xxxxxxxx]
                #extra_labels: [InstanceName]
                #statistics_types: [last]
                #period_seconds: 60
                #metric_name_type: 2

   3. Press the Esc key, enter :wq, and then press the Enter key. Save and exit the qcloud-redis-product.yml configuration file.

6. Run the following command to install the exporter and start the service:

   qcloud_exporter --config.file "qcloud-redis-product.yml"

Install an exporter on the host where the AWS service resides.

1. Log on to the host where the AWS service resides as the root user.

2. Run the following command to configure the environment variables:

   AWS_ACCESS_KEY_ID="AKIAZUF26NQPE****"

   AWS_SECRET_ACCESS_KEY="8mSpf+XRHBuJEi4tjJN4YU6h3LRS****"

3. Create a configuration file named example.yml for the AWS service.

   1. Run the following command to create an empty file named example.yml:

      touch example.yml

   2. Run the following command to edit the example.yml file:

      vi example.yml

   3. Edit the example.yml configuration file.

      region: us-east-2
      region: us-east-2
      metrics:
      - aws_namespace: AWS/EC2
        aws_metric_name: CPUUtilization
        aws_dimensions: [InstanceId]
        aws_dimenstion_select:
                InstanceID: [i-03c0a044201c5****,i-04e5e3be6f194****]
        aws_statistics: [Maximum]
      - aws_namespace: AWS/EC2
        aws_metric_name: DiskReadOps
        aws_dimensions: [InstanceId]
        aws_dimenstion_select:
                InstanceID: [i-03c0a044201c5****,i-04e5e3be6f194****]
        aws_statistics: [Maximum]
      - aws_namespace: AWS/EC2
        aws_metric_name: DiskWriteOps
        aws_dimensions: [InstanceId]
        aws_dimenstion_select:
                InstanceID: [i-03c0a044201c5****,i-04e5e3be6f194****]
        aws_statistics: [Maximum]
      - aws_namespace: AWS/EC2
        aws_metric_name: DiskReadBytes
        aws_dimensions: [InstanceId]
        aws_dimenstion_select:
                InstanceID: [i-03c0a044201c5****,i-04e5e3be6f194****]
        aws_statistics: [Maximum]

      Parameter	Description
      region	The region where the AWS service resides.
      aws_namespace	The namespace of the AWS service.
      aws_metric_name	The metrics for the AWS service.
      aws_dimensions	The dimension in which a metric for the AWS service is queried, for example, InstanceId.
      InstanceID	The instance ID of the AWS service.
      aws_statistics	The statistical method of the AWS service.

   4. Press the Esc key, enter :wq, and then press the Enter key. Save and exit the example.yml configuration file.

4. Run the following command to download the installation package of the exporter:

   wget https://cms-prometheus.oss-cn-hangzhou.aliyuncs.com/aws-cloudwatch/cloudwatch_exporter-0.9.0-jar-with-dependencies.jar

5. Run the following command to create the target directory:

   mkdir target

6. Run the following command to move the exporter installation package cloudwatch_exporter-0.9.0-jar-with-dependencies.jar to the target directory:

   mv cloudwatch_exporter-0.9.0-jar-with-dependencies.jar target

7. Run the following command to install the exporter and start the service:

   java -jar target/cloudwatch_exporter-0.9.0-jar-with-dependencies.jar 9106 example.yml

   If the following information is displayed, the exporter is installed.

   2021-06-11 03:24:55.743:INFO::main: Logging initialized @1775ms to org.eclipse.jetty.util.log.StdErrLog
   2021-06-11 03:24:55.844:INFO:oejs.Server:main: jetty-9.4.z-SNAPSHOT; built: 2019-02-15T16:53:49.381Z; git: eb70b240169fcf1abbd86af36482d1c49826****; jvm 1.8.0_282-b08
   2021-06-11 03:24:55.906:INFO:oejsh.ContextHandler:main: Started o.e.j.s.ServletContextHandler@3de8f619{/,null,AVAILABLE}
   2021-06-11 03:24:55.964:INFO:oejs.AbstractConnector:main: Started ServerConnector@3feb2dda{HTTP/1.1,[http/1.1]}{0.0.0.0:9106}
   2021-06-11 03:24:55.965:INFO:oejs.Server:main: Started @1997ms

Create a metric import task.

1. Log on to the Cloud Monitor console.

2. In the left-side navigation pane, choose Hybrid Cloud Monitoring > Namespace.

3. Find the namespace and click the icon for account quantity, cloud service quantity, metric quantity, or task quantity in the Metric Data column.

   Note

   Task quantity indicates the total number of tasks that are used to import metrics from self-managed data centers, AWS, Tencent Cloud, and Prometheus, on-premises log monitoring data, and Simple Log Service log monitoring data.

4. In the Metric management panel, click the other IDC tab. Then, click Metric Data.

5. In the Basic Task Information step of the Metric Data dialog box, configure the basic information of the metric import task and click Next.

   Parameter	Description
   Select Middleware to Be Monitored	The middleware you want to monitor.
   Namespace	The namespace for the data you are ingesting.
   Task Name	The name of the metric import task.
   Task Description	The description of the metric import task.
   Select Host	The host where the cloud service resides. Note You must make sure that the Cloud Monitor agent is installed on all the selected hosts.
   Region	The region where the host resides. Note You can click Advanced Options to set this parameter.
   Network Type	The network type of the host. Valid values: VPC classic Note You can click Advanced Options to set this parameter.

6. In the Data Access Check step, check whether the latest version of the Cloud Monitor agent and the exporter are installed, and click Next.

7. In the Task Details step, enter the URL of the collection target, select the interval and timeout period for data collection, set tags for the metric import task, and then click Next.

8. In the Dashboard step, click Close.

Log on to the Cloud Monitor console.

In the Metric management panel, click the other IDC tab. On the AWS or Tencent Cloud tab, find the metric import task that you want to manage and click Modify in the Actions column.

In the Basic Task Information step of the Metric Data dialog box, configure the basic information of the metric import task and click Next.

In the Data Access Check step, check whether the latest version of the Cloud Monitor agent and the exporter are installed, and click Next.

In the Task Details step, enter the URL of the collection target, select the interval and timeout period for data collection, set tags for the metric import task, and then click Next.

In the Dashboard step, click Close.

Log on to the Cloud Monitor console.