Platform For AI:Manage service groups

Scenarios

Create a service group

When you create a service, you can specify the service group to which the service belongs.

The following example shows how to create a service group named pmml and how to add the pmml_prod and pmml_grey services to the service group.

Create a service group in the PAI console

1. Go to the EAS-Online Model Services page.

   1. Log on to the Machine Learning Platform for AI console.

   2. In the left-side navigation pane, click Workspaces. On the Workspace list page, click the name of the workspace that you want to manage.

   3. In the left-side navigation pane, choose Model Deployment > Elastic Algorithm Service (EAS) to go to the EAS-Online Model Services page.

2. On the Group Service tab, click Create Group Service.

3. On the Deploy Service page, set the required parameters and click Deploy.

   Take note of the following parameters:

   • Service Name: Follow the on-screen instructions to specify a valid service name. Example: pmml_prod.

   • Group: the service group to which the service belongs. In this example, New Group is used, and the new group name is set to pmml.

   For information about other parameters, see Upload and deploy models in the console.

Repeat Steps 2 and 3 to create a service named pmml_grey that belongs to the pmml service group.

Create a service group by using a client

1. Prepare the service configuration file named service.json.

   • The following code provides an example on the configuration file content of the pmml_prod service:

     {
       "name":"pmml_prod",
       "model_path":"http://eas-data.oss-cn-shanghai.aliyuncs.com/models/lr_xingke4.pmml",
       "processor":"pmml",
       "metadata":{
         "cpu":1,
         "instance":4,
         "group":"pmml",
         "traffic_state": "grouping"
       }
     }

   • The following code provides an example on the content of the configuration file for the pmml_grey service:

     {
       "name":"pmml_grey",
       "model_path":"http://eas-data.oss-cn-shanghai.aliyuncs.com/models/lr_xingke4.pmml",
       "processor":"pmml",
       "metadata":{
         "cpu":1,
         "instance":1,
         "group":"pmml",
         "traffic_state": "grouping"
       }
     }

   Parameters:

   • group: the name of the service group that is created. This parameter specifies the service group to which the service belongs.

   • traffic_state: specifies whether to forward traffic to the service in the service group. Valid values:

     • grouping: forwards traffic to the service.

     • standalone: does not forward traffic to the service.

     Note

     If you do not configure the traffic_state parameter in the configuration file of a service, the system forwards traffic to the service by default. If you do not want a service that is added to a service group to receive traffic immediately, set traffic_state to standalone.

     If a service group contains only a single service and you set traffic_state to standalone for the service, the system automatically changes the value to grouping.

   For more information about other parameters in a configuration file, see Run commands to use the EASCMD client.

2. Create two services and a service group.

   Log on to the EASCMD client and run the create command to create two services and a service group. For more information about how to log on to the EASCMD client, see Download the EASCMD client and complete user authentication.

   $ eascmd create service.json

3. View information about the services and service group.

   Run the following ls command to query information about the services and service group:

   $ eascmd ls

   The following information is returned:

   [RequestId]: 716BEBFC-E8A4-51FD-A3F7-56376B167923
   +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+
   |        SERVICENAME        | INSTANCE | CPU | MEMORY |      CREATETIME      |      UPDATETIME      | STATUS  | WEIGHT | TRAFFICSTATE |       SERVICEGROUP        |
   +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+
   | pmml_prod                 |        4 |   1 | 1000M  | 2022-06-05T14:30:49Z | 2022-06-05T14:30:49Z | Running |     80 | grouping     | pmml                      |
   | pmml_grey                 |        1 |   1 | 1000M  | 2022-06-05T14:31:38Z | 2022-06-05T14:31:38Z | Running |     20 | grouping     | pmml                      |
   +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+

   Parameters:

   • pmml is displayed in the SERVICEGROUP column. This indicates that the two services belong to the pmml service group.

   • grouping is displayed in the TRAFFICSTATE column. This indicates that both services receive traffic. The percentages of the traffic that the services receive are 80% and 20%, which are calculated based on the number of service instances.

What to do next:

• After you create the services and service group, you can view the data ingress of the service group and the data ingresses of services. The data ingresses are used for external access. For more information, see View data ingresses.

• After you create the services and service group, the system uses group traffic switchover by default. The percentage of traffic that each service receives is based on the number of service instances. You can change the traffic switchover method and modify the traffic distribution policies. For more information, see Modify traffic distribution policies (group traffic switchover and blue-green traffic switchover).

View data ingresses

Modify traffic distribution policies (group traffic switchover and blue-green traffic switchover)

Elastic Algorithm Service (EAS) supports group traffic switchover and blue-green traffic switchover. After a service is created, group traffic switchover is used by default. You can change group traffic switchover to blue-green traffic switchover. You can also modify traffic distribution policies for the two traffic switchover methods.

Group traffic switchover

• Introduction

  After you create services and specify the service group to which the services belong, group traffic switchover is used by default. The services in the service group start to receive traffic, and the percentage of traffic that each service receives is determined based on the number of service instances.

• Modify traffic distribution policies

  • To modify traffic distribution policies, follow the instructions shown in the following figure in the PAI console. If you turn on the switch indicated by ④, the service receives traffic. If you turn off the switch, the service does not receive traffic.

  • Run the following release command to modify traffic distribution policies:

    $ eascmd release <service_name> -s grouping|standalone

    Parameters:

    • <service_name>: the name of the service. Change the value to the name of the service for which you want to modify the traffic distribution policy.

    • grouping|standalone: the state after modification. Valid values: grouping and standalone. grouping indicates that the service receives traffic. standalone indicates that the service does not receive traffic.

    Examples:

    • Run the following command to change the state of the pmml_grey service to standalone. This way, the pmml_grey service does not receive traffic.

      $ eascmd release pmml_grey -s standalone

      The following output is returned:

      Confirmed to release service [pmml_grey] to group traffic [Y/n]yes
      [RequestId]: 40C787DF-8900-5F7A-8A01-30F7D5A8BF3B
      [OK] Service [pmml_grey] has entered the traffic state: standalone

      Run the eascmd ls command to query the state of the service. The following output is returned:

      [RequestId]: 83BE3FBB-8CE2-5008-B435-1938A20B13AA
      +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+
      |        SERVICENAME        | INSTANCE | CPU | MEMORY |      CREATETIME      |      UPDATETIME      | STATUS  | WEIGHT | TRAFFICSTATE |       SERVICEGROUP        |
      +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+
      | pmml_prod                 |        4 |   1 | 1000M  | 2022-06-05T14:30:49Z | 2022-06-05T14:30:49Z | Running |    100 | grouping     | pmml                      |
      | pmml_grey                 |        1 |   1 | 1000M  | 2022-06-05T14:42:41Z | 2022-06-05T14:42:41Z | Running |     0  | standalone   | pmml                      |
      +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+

      The TRAFFICSTATE of the pmml_grey service changes to standalone. The value of the WEIGHT parameter is 0. This indicates that all traffic is received by the pmml_prod service.

    • Run the following command to change the state of the pmml_grey service to grouping. This allows the pmml_grey service to receive traffic.

      $ eascmd release pmml_grey -s grouping

      The following output is returned:

      Confirmed to release service [pmml_grey] to group traffic [Y/n]yes
      [RequestId]: 40C787DF-8900-5F7A-8A01-30F7D5A8BF3B
      [OK] Service [pmml_grey] has entered the traffic state: grouping

      Run the eascmd ls command to query the state of the service. The following output is returned:

      [RequestId]: 83BE3FBB-8CE2-5008-B435-1938A20B13AA
      +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+
      |        SERVICENAME        | INSTANCE | CPU | MEMORY |      CREATETIME      |      UPDATETIME      | STATUS  | WEIGHT | TRAFFICSTATE |       SERVICEGROUP        |
      +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+
      | pmml_prod                 |        4 |   1 | 1000M  | 2022-06-05T14:30:49Z | 2022-06-05T14:30:49Z | Running |     80 | grouping     | pmml                      |
      | pmml_grey                 |        1 |   1 | 1000M  | 2022-06-05T14:42:41Z | 2022-06-05T14:42:41Z | Running |     20 | grouping     | pmml                      |
      +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+

      The TRAFFICSTATE of the pmml_grey service changes to grouping. The percentage of traffic that is received by the service is 20%.

Blue-green traffic switchover

• Introduction

  You can use blue-green traffic switchover only if you create two services in a service group. If one service in the service group receives N% of traffic, the other service receives (100 - N)% of traffic. If you delete a service in the service group, the other service receives all traffic.

• Modify traffic switchover methods

  If group traffic switchover is used, you can run the following command and change the traffic switchover mode to blue-green traffic switchover.

  Important

  Group traffic switchover and blue-green traffic switchover are mutually exclusive. After you use blue-green traffic switchover, you cannot change it to group traffic switchover.

  $ eascmd release <service_name> -w <weight>

  Parameters:

  • <service_name>: the name of the service. Change the value to the name of the service for which you want to modify the traffic switchover mode.

  • <weight>: the weight of the service. Valid values: 0 to 100.

  The following section describes how to modify traffic distribution policies.

• Modify traffic distribution policies

  You can specify the weights of services by using blue-green traffic switchover. Run the following command to specify the weight of a service:

  $ eascmd release pmml_grey -w 27

  The following output is returned:

  Confirmed to release service [pmml_grey] at weight [27%]? [Y/n]yes
  [RequestId]: DD8F8EC2-D726-5E37-9C0B-B360F61C810A
  [OK] Service [pmml_grey] is weighted to 27% now

  Run the eascmd ls command to query the state of the service. The following output is returned:

  [RequestId]: 51789E0B-88E7-5293-828B-1EFAD2EEA003
  +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+
  |        SERVICENAME        | INSTANCE | CPU | MEMORY |      CREATETIME      |      UPDATETIME      | STATUS  | WEIGHT | TRAFFICSTATE |       SERVICEGROUP        |
  +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+
  | pmml_prod                 |        4 |   1 | 1000M  | 2022-06-05T14:30:49Z | 2022-06-05T14:53:20Z | Running |     73 | blue-green   | pmml                      |
  | pmml_grey                 |        1 |   1 | 1000M  | 2022-06-05T14:50:14Z | 2022-06-05T14:53:20Z | Running |     27 | blue-green   | pmml                      |
  +---------------------------+----------+-----+--------+----------------------+----------------------+---------+--------+--------------+---------------------------+

  The TRAFFICSTATE of the pmml_grey service changes to blue-green, which indicates that the blue-green traffic switchover is used. The WEIGHT of the service is 27, which indicates that 27% of traffic is received by the pmml_grey service.